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ABSTRACT 

A study examined the system of performance measures 
and standards approved in each state in response to the Carl D. 
Perkins Vocational and Applied Technology Act of 1990. An initial 
letter and two follow-up letters were sent to j4 state directors of 
education. All states responded with information 
thei~ approved systems of measures for secondary 



vocat ional 
concerning 

vocational education programs and 
concerning their approved systems 
programs. Findings indicated that 
and 19 at the postsecondary level 
performance measures and standards 



52 responded with information 
of measures for postsecondary 
16 states at the secondary level 
developed separate systems of 
for assessing basic and advanced 
academic skills. In the assessment of secondary academic skills, high 
school proficiency exams were a commonly used technique for assessing 
basic academics; portfolios and course/program completion were used 
more frequently to assess advanced academics. About two-thirds of the 
states at the secondary level used something other than nationally 
recognized techniques for assessing academic skills. Work skill 
attainment was assessed predominantly by locally selected or 
developed techniques at both the secondary and postsecondary level. 
The most frequently reported standard for secondary program 
completion was 80-90 percent. Locally selected and course/program 
completion were the most popular techniques for assessing academic 
skills and competency attainment at the postsecondary level. 
(Contains 13 references.) (YLB) 
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FOREWORD 



Each state has been required to establish a system of performance measures and standards 
for use in program evaluation and improvement by the Carl D. Perkins Vocational and Applied 
Technology Education Act of 1990. Information about these approved systems has been limited 
A previous report by these authors (McCaslin and Headley, 1993) reported on the measures 
included in these approved systems. This report addresses the standards and assessment 
techniques included in these systems. 

Funding for the study was provided by the Graduate School at The Ohio State University 
through a seed grant for the Comprehensive Vocational Education Program Special appreciation 
is extended to Dr. Roy A. Koenigsknecht, Dean of the Graduate School for this support. 

The project staff is indebted to the State Directors of Vocational Education and their staff 
who provided the information on their approved system of performance measures and standards. 
Without their help this study could not have been completed 

The researchers are also grateful to the following members of the Coordinating Council 
for the Comprehensive Vocational Education Program Drs. David Boggs, Sharon V. Redick, 
and Michael Scott. The Comprehensive Vocational Education Program is an interdepartmental 
effort of three colleges. These colleges and the Deans include: Education, Dr. Nancy L. 
Zimpher; Food, Agricultural, and Environmental Sciences, Dr. Bobby D. Moser; and Human 
Ecology, Dr. Jerelyn Schultz. Ms. Connie Myers and Ms. Doris Kislci provided secretarial 
services to the project. A final note of appreciation is extended to Mr. Greg Belcher who assisted 
with the final editing and production of this document. 

R. Kirby Banick 
Professor and Chair 
Coordinating Council for 

Comprehensive Vocational Education 
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INTRODUCTION 



The move to adopt performance measures and standards for vocational education can be 
viewed as a response to the calls for educational reform that have occurred in the last decade in 
The United States. The use of performance standards in the Job Training Partnership Act (JTPA) 
programs also influenced the reauthorization of the Carl D. Perkins Vocational Education Act 
of 1984. 

The statewide systems of performance measures and standards mandated by the Carl D. 
Perkins Vocational and Applied Technology Education Act of 1990 (here after referred to as 
Perkins 1990) were to be approved and in place by September, 1992. As reported by McCaslin 
and Headley ( 1993), the states took the mandate seriously. By April of 1993, ninety-six percent 
of the states reported academic measures and standards for secondary vocational programs, while 
ninety-two percent reported academic measures and standards for postsecondary programs. For 
other performance, one hundred percent of the states had approved at least one measure for 
secondary programs, while ninety-six percent of the states had approved at least one measure for 
postsecondary programs. More complete information regarding the nature of measures approved 
by the states for use in assessing vocational education programs was previously repented in A 
National Study of Approved State Systems of Performance Measures and Standards for 
Vocational Education (McCaslin and Headley, 1993). 

Regarding measures and standards, "the legislative history of the Act indicates that a 
measure is a description of an outcome, and a standard is the level or rate of that outcome" 
(Center for Law and Education, 1990). Once a state decided on the types of academic and other 
performance to assess (measures), it was then necessary to decide on the level of acceptable 
performance (standards). As noted by Hoaehlander, Levesque, and Rahn ( 1992) and Center for 
Law and Education (1990), developing standards for performance of vocational education 
programs was no simple matter. Further, it was necessary for states to select or develop 
appropriate assessment instruments and procedures. 

This report is the result of a national study of approved state systems of performance 
measures and standards. The sections that follow contain a discussion of the advent of student 
outcomes assessment in vocational education, a review of the influence of the JTPA experience 
with performance standards on vocational education programs, development of performance 
standards, and selection of appropriate assessment techniques. 

Student Outcomes Assessment in Vocational Education 

Reauthorization of federal vocational education legislation occurred at a time when much 
national attention was being focused on the relationship between education and the future of the 
American economy. Concerns over the quality of vocational education were noted, and as the 
Office of Technology Assessment (OTA) reported, "one manifestation of the concern for quality 
of vocational programs has been the interest in using outcome measures as indicators of program 
effectiveness" (OTA, 1989, p. 2). OTA investigated the use of outcome measures, including 
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student labor-market outcomes and student competency, and found that measures of this type 
were in use in a number of states. 

External and internal pressures led to the adoption of student outcomes assessment in 
vocational education. Bragg and Harmon (1992) described several influences, in postsecondary 
vocational education that led to the use of student outcomes assessment. Principally, external 
pressures such as public opinion, federal and state legislation, and certification and accreditation 
systems required more accountability. Internal pressures, such as faculty and student opinion, 
and stakeholder concerns, as well as administrative considerations for resource allocation dictated 
outcomes data be used for program improvement purposes. 

Stecher and Hanser (1992), noted that a number of secondary and postsecondary 
vocational institutions already had accountability systems in place before the implementation of 
Perkins 1990. The requirements of performance measures and standards have required many of 
these accountability systems to change. Previously, emphasis had been on evaluation of input 
indicators such as teacher credentials, size of facilities, teacher/student ratios, and adequacy of 
equipment. Although the examination of these types of indicators may not have been totally 
abandoned, the mandated use of performance measures and standards for student outcomes has 
brought a new focus in assessing the quality of vocational programs. Several changes in focus 
have been noted as Perkins 1990 requirements were incorporated into state and local 
accountability systems. Bragg and Harmon ( 1992) found, "Postsecondary institutions are shifting 
away from measuring inputs to determine the quality of their services to measuring outcomes 
such as learner knowledge and skill attainment" (p. 5). 

Concerns over the validity of traditional assessment techniques have arisen as local 
institutions and state agencies moved to adopt student outcomes assessment. The need for 
expanding existing instruments grew as vocational educators recognized the value of alternative 
assessments and the need for documenting skills not easily assessed by paper and pencil tests. 
Therefore, the shift in emphasis in program evaluation has also prompted a reevaluation of how 
student performance in vocational education programs is assessed With the increased scrutiny 
of student assessment in vocational education has come an increased examination of the 
procedures and practices used for assessing student performance. As pointed out by Whichard 
and Cobb (1993), there is £ close relationship to student assessment, curriculum, and standards. 
That is, if there is a desire to raise standards, this must be accompanied by curricular changes, 
which in turn must be coupled with "relevant measures of student achievement" (p. 5). 

The move towards performance standards for vocational education based on student 
outcomes brought with it a need for accurate, reliable and valid techniques for assessing student 
performance. Two types of student outcomes were to be assessed, academic and occupational. 
Perkins 1990 not only introduced the use of performance measures and standards in vocational 
education as national policy, but also legislated the integration of academic and vocational skills. 

Assessment of student outcomes in both academic and occupational competence were 
mandated by Perkins 1990. Though a variety of assessment methods have bcsn available and 
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are continually being developed, the Office of Technology Assessment (1992) reported that 
ideally all performance assessment techniques display the following characteristics: 

* they require students to construct responses rather than select from a set of 
-answers; 

* they assess behaviors of interest as directly as possible; 

* they are in some cases aimed at assessing group performance rather than 
individual performance; 

* they are criterion-referenced, meaning they provide a basis for evaluating a 
student's work with reference to criteria for excellence rather than with reference 
to other students work; 

* in general they focus on the process of problem solving rather than just on the end 
result; 

* carefully trained teachers or other qualified judges are involved in most of the 
evaluation and scoring; and 

* students understand clearly the criteria on which they are judged" (p. 19-20). 

The use of student performance assessment in vocational education programs is not new. 
What is new, however, is the reliance on these assessments as a means of program evaluation. 
The mandate for performance measures and standards included in Perkins 1990 is one of a 
number of indicators that the current and future emphasis in evaluation of vocational education 
programs will center cm student outcomes. This emphasis will bring with it the need for carefully 
developed standards and reliance on accurate and reliable techniques for assessing student 
performance. 

The JTPA Experience with Performance Standards 

The precursor for the use of performance standards in vocational education programs was 
the Job Training Partnership Act. Butler (1988) stated, "in general there has been great 
consistency over the several years of JTPA, providing a good basis of experience with outcome 
measures for a national program" (p. 2). Added as an introductory remark to a study evaluating 
the effects of performance standards on JTPA clients, Dickinson, West, Kogan, Drury, Franks, 
Schlichtmann, and Vencill (1988) reported that, H many of the findings can be helpful when 
enacting legislation for other human resource programs" (p. iv). Programs funded under JTPA 
have been required, since 1984, to be evaluated on the basis on performance standards. These 
standards were based on client outcomes, such as placement rates, wages and costs per placement. 
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The strengths of the performance standards for JTPA, in the minds of policy makers are 
in their use as a mechanism for program accountability, and as a public relations device. In fact, 
as reported by Dickinson, et al. ( 1988), the standards were implemented to achieve the following 
purposes: 

* . To hold service delivery areas (SDAs) accountable for program outcomes, 

* To encourage the achievement of quality employment outcomes, 

* To encourage the achievement of cost-effective outcomes, 

* To create an incentive for effective management by local program administrators, 
and 

* To foster acceptance of the program by the business community (p. 1). 

TTie findings of the study by Dickinson, et al. (1988) revealed that performance standards 
had both intended and unintended effects. The unintended effects noted by the study included, 
in some cases, serving fewer of the hard-to-serve clients and the delivery of less intense 
programs. However, the study also stated that unintended effects were generally associated with 
a lack of well defined goals in service delivery areas. The authors concluded that performance 
standards were having the desired effect on JTPA services and any unintended effects were not 
large or inevitable. 

Hie measures of performance developed by the Department of Labor for JTPA programs 
as reported by Butler (1988) for adults included: entered employment rate, cost per entered 
employment rate, average wage at placement, and welfare entered employment rate. The youth 
measures included entered employment rate, positive termination rate, and cost per positive 
termination. The standards were set, "so that about 75% of all SDAs achieve them, and adjusted 
upward or downward for each new year depending upon the previous year's experience' 1 (Butler, 
1988, p. 5). Butler investigated the implications of the JTPA experience with performance 
standards for vocational education and concluded that: 

JTPA has demonstrated that it is possible to establish a national program substantially 
driven by clearly-specified, measurable outcomes. Moreover, it is possible to tie those 
outcomes to funding, not only at the federal and state levels, but even down to the level 
of actual service delivery area through performance-based contracts with educational 
institutions or community agencies. As policies which will underlie vocational education 
are developed, JTPAs experience should encourage planners that a focus on outcomes can 
be achieved (p. 21). 

The data on the effects of performance standards in JTPA programs supported the desire 
to implement national policy that would hold vocational education programs accountable for 
student outcomes. Concerning a system of performance standards, Butler (1988) stated, "With 
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appropriate modifications for difference of purpose, it ought to be emulated in large part by 
revised vocational education legislation' (p. 3). As noted, the JTPA experience with performance 
standards led the way to their use in vocational education. The following section will present 
information related to the development of performance standards for use with vocational 
education programs. 

Development of Performance Standards 

The U.S. Department of Education defined a measure as "a description of an outcome" 
and standard as "the level or rate of an outcome" (Federal Register, August 14, 1992, p. 36728). 
An outcome is defined as "a measurable aspect of student performance" (Hoachlander, Levesque, 
and Rahn, 1992, p. 42). Taking an example from the Perkins Act, academic skill is an outcome, 
mathematics achievement on a standardized test is a measure and 80% of the students attaining 
a score of 80% on the test is a standard 

It should be noted that there is no one right way to develop standards, but in fact a 
number of ways to develop standards. Standard setting involves judgement. Testing in American 
Schools: Asking the Right Questions, (OTA, 1992) pointed out that judgements concerning 
standards should be made by persons who are qualified to make them, meaningful to those who 
make them and reflecting the purpose for adopting the standards. Perkins 1990 required that each 
state appoint a Committee of Practitioners which would "review, comment on, and propose 
revisions to a draft State proposal, which the State board shall develop, for a system of core 
standards and measures of performance for vocational programs" (sec 1 15a). This Committee 
was to act as a check to ensure that the standards selected were satisfactory. 

Setting standards for student performance is a challenging task. When standards for 
performance not only reflect student performance but also that of the vocational program, care 
must be taken to ensure that they reflect levels of performance actually required in the world of 
work. What then should be considered in developing standards. Hoachlander, Levesque, and 
Rahn (1992) describe four sources of information to be used in developing standards. These 
were: departure from averages, industry standards, best standards, and certification and licensing 
requirements. Departure from averages involved using information gathered from standardized 
tests, or from average scores obtained from assessment measures put in place as part of the state's 
system of measures and standards. Industry standards could be used for those industries that have 
developed performance standards. Best practices referred to standards that have already been 
adopted by vocational institutions. Finally, levels of performance on certification or licensing 
exams could be selected as a standard of performance in some vocational programs. 

Regardless of the source of information related to the development of standards, several 
points must be kept in mind First, standards must reflect actual levels of performance needed 
to succeed in the world of work. Second, states would have an opportunity to adjust standards 
depending on their efficacy in achieving intended results. Finally, standards should be developed 
through a sound, reasoned approach, and not derived arbitrarily. 



Given careful development of a system of performance measures and standards, another 
step in implementation is the selection or development of appropriate assessment techniques. The 
following section addresses the development of assessment techniques. 

Assessment Techniques 

Student assessment has been used to ascertain the level of student performance in 
vocational programs since its inception. The traditional function of assessment in vocational 
programs was to certify student competence in an occupational area While assessment still plays 
this role, assessment data also provides feedback for teachers in modifying their instruction, and, 
more recently, for program accountability purposes. If the goal of a vocational program was to 
prepare persons for the world of work, then the composite picture of what these persons have 
achieved is viewed as an indicator of program quality. 

Recent research has focused on the relationship between assessment and student 
achievement, and assessment and curricular modification. In particular, with the need for 
vocational programs to show academic gains for students, assessment techniques for academic 
skill, in addition to occupational competence, were required to be developed Traditionally, 
academic skills have been assessed with standardized tests. Whichard and Cobb (1993) pointed 
out the deficiencies with these tests. Among these were: standardized tests ignore the process 
and focus on outcomes only; standardized tests are poor predictors of student potential; the 
validity of many of the tests has been seriously questioned; and standardized tests only measure 
a narrow range of knowledge and behavior. Whichard and Cobb suggested four alternative 
assessments, which included: portfolios, exhibition assessment, dynamic assessment, and 
curriculum-based assessment. It is proposed that these types of assessment address the questions 
raised with standardized tests. 

There are certainly problems associated with the use of these types of assessment 
techniques for vocational education students, including the time and cost involved and the general 
unfamiliarity of educators with these assessments. In addition to these assessments of student 
performance, Bragg and Harmon (1992) identified a number of techniques already in use by 
postsecondary vocational education institutions for assessing outcomes. Among these were: 
standardized tests, exams graded by panels of experts, faculty and student surveys, transcript 
analyses, and state unemployment and institutional placement records. As reported by Bragg and 
Harmon (1992), a number of different techniques were in place to measure student outcomes, 
including standardized tests and alternative assessments of student academic and occupational 
competence. 

Perkins 1990 permitted flexibility in deciding on types of techniques to be used by states 
in assessing outcomes of vocational students. The thrust of the legislation was not to address 
methods, but outcomes. States needed to select techniques whose results could be used in 
satisfying the data needs of the federal government. A number of states had student assessment 
programs in place for academic achievement. In many of these cases, data collected from these 
systems could be adapted to satisfy the requirements of the newly implemented systems of 



measures and standards. In other cases, where no statewide system of student assessment 
existed, it was necessary to devise or select assessment techniques for the academic competence 
of vocational students. 

PROBLEM STATEMENT 

The move to implement a system of performance measures and standards in vocational 
education is a large undertaking. The states had until September 25, 1992 to implement the 
systems of standards and measures (Federal Register, August 14, 1992). Many of the states 
found it necessary to develop new evaluation procedures as a result of the mandates of the Carl 
D. Perkins Vocational and Applied Technology Education Act of 1990. Yet, relatively litde 
information was available regarding this process. Hoachlander and Rahn (1992) gathered 
information in 1991 from the states in an effort to determine the expected makeup of the systems. 
However, as stated by Hoachlander and Rahn, "The systems actually implemented in fall 1992 
may look substantially different, as states continue to develop performance measures and 
standards" (p. 2). McCaslin and Headley (June 1993) reported findings related to the 
performance measures that had been approval by the states for use with their statewide systems 
of performance measures and standards. Information concerning the standards and techniques 
used in assessing performance was also gathered in the McCaslin and Headley study. This type 
of information is needed in making comparisons across the states in order to further develop and 
improve the existing system of performance measures and standards and in meeting the 
requirements of the Carl D. Perkins Vocational and Applied Technology Education Act of 1990. 

The measures of performance, standards for that performance and techniques selected to 
assess performance all work together in a statewide system of performance measures and 
standards. Information has been reported concerning the measures approved by the states. In 
order to get a more complete picture of these systems, information is also needed on the 
standards and assessment techniques. 

PURPOSE AND OBJECTIVES 

This study was sponsored by the Graduate School at The Ohio State University through 
a seed grant for the Comprehensive Vocational Education Program The purpose of this study 
was to examine the system of performance measures and standards that had been approved in 
each state of the United States in response to the Carl D. Perkins Vocational and Applied 
Technology Act of 1990. The specific objectives addressed in this paper were: 

1. To determine what standards for academic performance had been approved in each 
state. 

2. To determine what types of assessment techniques had been approved for 
academic performance in each state. 
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3. To determine what standards for other performance had been approved in each 
state. 

4. To determine what types of assessment techniques had been approved for other 
■ -performance in each state. 

METHODOLOGY 

Descriptive-survey and content analysis research methods were used in this study. The 
names and addresses of the 54 stare directors of vocational education were used to generate the 
population for this investigation. For the purposes of this study, a state was defined as including 
all 50 states in the United States and the District of Columbia, Puerto Rico, Guam, and The 
Virgin Islands. 

An initial letter (see Appendix A) was sent to each state director on November 24, 1992 
requesting documents describing the system of performance measures and standards that had been 
approved by their state board for vocational education. This strategy was used to minimize the 
amount of time and energy required to provide the information. Approximately four weeks later, 
a follow up letter, containing the original request, was sent to the 19 state directors that had not 
responded A third and final follow-up letter was sent on February 1, 1993 to the remaining 
seven state directors that had not responded On March 3, 1993 phone calls were made to the 
remaining five state directors from which no response had been received As of April 15, 1993, 
all of the states (100%) had responded with information concerning their approved systems of 
measures for secondary vocational education programs and fifty-two (96%) had responded with 
information concerning their approved systems of measures for postsecondary programs. Two 
states (Iowa and The Virgin Islands) indicated that their system of measures and standards had 
not been approved by their state boards. Additionally, Georgia and Arkansas did not report their 
system of postsecondary performance measures and standards. 

Once the documents had been reviewed, their content was analyzed The analysis resulted 
in a listing, by states, of the standards that the states had adopted, using the categories listed in 
section 1 15 of the Carl D. Perkins Vocational and Applied Technology Education Act of 1990 
(a) basic academic skill, (b) advanced academic skill, (c) competency attainment, (d) work skill 
attainment, (e) retention/completion, (f) placement, (g) service to special populations, and (h) 
other measures. In addition to the standards, techniques used to assess performance were 
identified for each category listed above. 

A summary sheet was developed for each state, presenting the data on adopted measures. 
On March 12, i993, die summary sheets were mailed to the state directors of vocational 
education (a copy of the summary sheet, instructions and the cover letter are contained in 
Appendix B). Personnel from each state were asked review, verify and amend the listing as 
necessary. When discrepancies occurred, a further review of the documents was carried out. If 
necessary, a follow up call was made to the state director of vocational education's office for 
additional clarification. 

8 



FINDINGS 



Secondary Vocational Education Standards 

This section reports the secondary vocational education standards approved by states 
according to the major categories required by The Carl D. Perkins Vocational and Applied 
Technology Education Act Amendments of 1990. These categories included: academic and other 
performance. 

Academic Standards 

Academic standards were categorized in two groups: basic and advanced Fifty-eight 
percent of the states reported the use of one set of measures and standards for both basic and 
advanced academics (McCaslin and Headley, 1993). Table 1 presents information concerning 
the number of states with approved standards for these measures. Standards were reported for 
the academic areas of reading, mathematics, language, science and other academic skills. The 
most frequently reported basic academic standards were in mathematics (85%), reading (80%), 
and language (77%). The most frequently reported advanced academic standards were in 
mathematics (76%), reading (65%), and language (65%). 

Table 1 

A Summary of Approved State Standards for Academic Skills (Secondary) (N=54) 



Academi: Skill f % 



Basic 



Reading 43 80 

Language 40 77 

Mathematics 46 85 

Science 19 35 

Other 24 44 

Advanced 

Reading 35 65 

Language 35 65 

Mathematics 41 76 

Science 22 41 

Other 23 43 
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In addition to examining the standards which had been approved by the states for use with 
academic performance measures, this study also examined the types of techniques used in 
assessing performance. Four major types of techniques were identified: state developed, local 
selected or developed, nationally recognized, and other. State developed techniques included high 
school proficiency exams (e.g., The Ohio Ninth Grade Proficiency Test), and other state 
assessment programs (e.g., The Louisiana Education Assessment Program). Local developed 
techniques were those developed or selected by a local education agency. In some cases the state 
suggested some nationally recognized instrument for possible use by local education agencies. 
In other states, the choice of technique was left totally to the local agency. Nationally recognized 
techniques were those developed for use across states and in many cases had national norms (e.g., 
Iowa Test of Basic Skills). Other techniques included gain or progress in a course/program, or 
completion of course work. 

A number of states reported the use of more than one assessment technique for each of 
the academic measures. For assessing reading, twelve states reported the use of more than one 
technique. Eleven states reported using more than one assessment technique for mathematics; 
ten states reported the use of more than one technique for language, seven states reported the use 
of more than assessment technique for science, and sue states reported the use of more than one 
technique for assessing other academic skills. 

Reading 

In the academic area of reading, forty-three states reported having approved standards for 
basic reading performance and thirty-five states reported approved standards for advanced reading 
performance (see Table 1). Information concerning the techniques used to assess reading 
performance and the standards for basic ?nd advanced reading performance is contained in this 
section. 

Basic Reading Performance. Table 2 presents information on the techniques used to 
assess basic reading performance. About one-half (47%) of the states reported the use of state 
developed assessment techniques. Thirty-seven percent of the states reported the use of 
nationally recognized techniques. Nine different instruments were reported to be in use, with the 
Iowa Test of Basic Skills, Test of Adult Proficiency, Comprehensive Test of Basic Skills, 
Stanford Achievement Test, and Test of Adult Basic Education being used by more than one 
state. Over one-fourth of the states (28%) reported the use of local developed or selected 
techniques, while 12% reported using other techniques, such as course completion or General 
Educational Development (GED) criteria 

Standards approved for use with measures of basic reading performance are displayed in 
Table 3. Of those states reporting the use of state developed assessment techniques, the 
percentage of students receiving a passing score was the most frequently reported (50%) standard 
Of the states reporting the use of a local selected technique 58% used gain scores, while the most 
frequently reported standard for those using a nationally recognized technique were gain scores 
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(37%). Over three-fourths (80%) of the states which employed other techniques for assessing 
basic reading performance reported course or program completion. 

Advanced Reading Performance. Table 4 presents information on the techniques used to 
assess advanced reading performance. One-third (34%) of the states reported the use of state 
developed assessment techniques. Thirty-seven percent of the states reported the use of 
nationally recognized techniques. Ten different instruments were reported to be in use; with the 
Stanford Achievement Test and the Test of Adult Basic Education being used by more than one 
state. Another one-third of the states (34%) reported the use of local developed or selected 
techniques, while one-fourth (25%) reported using other techniques, such as course/program 
completion and portfolios. 

Standards for techniques approved for use with measures of advanced reading performance 
are displayed in Table 5. Of those states reporting die use of state developed assessment 
techniques, the most frequently reported standards were percentage of students receiving a passing 
score (33%) and gain scores (33%). Of the states reporting the use of a local selected technique 
68% used gain scores. The most frequently reported standard for those using a nationally 
recognized technique was also gain scores (46%). Two-thirds (67%) of the states which 
employed other techniques for assessing advanced reading performance reported using a 
course/program completion standard 
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Table 2 

Techniques Used to Assess Basic Reading Performance (Secondary) (N=43) 



Assessment Techniques f % 



State Developed 20 47 

State High School Proficiency Exam 10 

Other State Developed Assessment 10 

Local Selected or Developed 12 28 

Nationally Recognized 16 37 

Iowa Test of Basic Skills 3 

Test of Adult Basic Education 3 

Test of Adult Proficiency 2 

Comprehensive Test of Basic Skills 2 

Stanford Achievement Test 2 

Metropolitan Achievement Test 1 

Stanford 8 3R Battery 1 

ACT Work Keys 1 

Gates-MacGinite Reading Test I 

Other 5 12 



Course/Program Completion 4 
GED Criteria 1 



12 



20 



Table 3 

Standards for Techniques Used to Assess Basic Reading Performance (Secondary) (N=43) 



Type of Techniques and Standards f % 



State Developed Techniques 20 
Standards 

% of Students Passing 10 50 

Grade Level/Norm Reference Score 2 10 

Gain Scores 3 15 

Comparison with all Students 3 15 

Local Program Determined 1 5 

To be Determined at State Level 1 5 

Local Selected or Developed Techniques 12 

Standards / 

Grade Level/Norm Reference Score 1 8 

Gain Scores 7 58 

Local Program Determined 4 33 

Nationally Recognized Techniques 16 

Standards 

National Norm Score 4 25 

Gain Scores 6 37 

Comparison with all Students 2 13 

Local Program Determined 1 6 

To be Determined at State Level 1 6 

Not Indicated 2 13 

Other Techniques 5 

Standards 

Gain or Progress 1 20 

Course/Program Completion 4 80 
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Table 4 

Techniques Used to Assess Advanced Reading Performance (Seconda ry) (N=35) 



Assessment Techniques 



f 



% 



State Developed 



12 



34 



State High School Proficiency Exam 
Other State Developed Assessment 



2 
10 



Local Selected or Developed 



12 



34 



Nationally Recognized 



13 



37 



Iowa Test of Basic Skills 1 

Test of Adult Proficiency 1 

Test of Adult Basic Education 3 

Comprehensive Test of Basic Skills 1 

Metropolitan Achievement Test 1 

Stanford Achievement Test 2 

Stanford 8 3R Battery 1 

ACT Work Keys 1 

Gates-MacGinite Reading Test 1 

VTECS Item Banks 1 

Other 9 25 

Course/Program Completion 6 

Portfolios 2 

Grade Advancement 1 



14 



22 



Table 5 

Standards for Techniques Used to Assess Advanced Reading (Secondary) (N=35) 



Type of Techniques and Standards f % 



State Developed Techniques 12 
Standards 

% of Students Passing 4 33 

Grade Level/Norm Reference Score 1 8 

Gain Scores 4 33 

Comparison with all Students 2 17 

To be Determined at State Level 1 8 

Local Selected or Developed Techniques 12 

Standards 

Grade Level/Norm Reference Score 2 16 

Gain Scores 8 68 

Local Program Determined 2 16 

Nationally Recognized Techniques 13 



Standards 

National Norm Score 3 23 

Gain Scores 6 46 

Comparison with all Students 1 8 

Local Program Determined 1 8 

To be Determined at State Level 1 8 

Not Indicated 1 8 

Other Techniques 9 

Standards 

Gain or Progress 1 1 1 

Course/Program Completion 6 67 

Not Indicated 2 22 



Language 



Forty states reported assessing basic language performance, while thirty-five states 
reported assessing advanced academic performance (see Table 1). This section contains 
information regarding the techniques and standards used to assess language performance. 

Basic Language Performance. Table 6 presents information on the techniques used to 
assess basic language performance. About one-half (47%) of the states reported the use of state 
developed assessment techniques. One-third (33%) of the states reported the use of local selected 
or developed techniques. About one-third of the states (30%) reported the use of nationally 
recognized techniques. Eight different instruments were reported to be in use, with the Iowa Test 
of Basic Skills, Test of Adult Basic Education, and Comprehensive Test of Basic Skills being 
used in more than one state. Ten percent of the states measuring basic language performance 
reported using course/program completion for the other assessment technique. 

Table 6 



Techniques Used to Assess Basic Language Performance (Secondary) (N=40) 



Assessment Techniques 


f 




% 


State Developed 




18 


47 


State High School Proficiency Exam 
Other State Developed Assessment 


10 
8 






Local Selected or Developed 




13 


33 


Nationally Recognized 




12 


30 


Iowa Test of Basic Skills 
Test of Adult Proficiency 
Test of Adult Basic Education 
Comprehensive Test of Basic Skills 
Metropolitan Achievement Test 
Stanford Achievement Test 
Stanford 8 3R Battery 
ACT Work Keys 


2 
1 
3 
2 
1 
1 
1 
1 






Other 

Course/Program Completion 


4 


4 


10 



Standards approved for use with measures of basic language performance are displayed 
in Table 7. Of those states reporting the use of state developed techniques, the percentage of 
students receiving a passing score was the most frequently reported (50%) standard Gain scores 
were the most frequently (69%) used standard for local selected or developed techniques. Gain 
scores were also the most frequently reported (39%) standard for nationally recognized 
assessment techniques. An additional 30% of these states reported using national norm scores 
as standards. All of the states reporting other techniques for assessing basic language used a 
course/program completion standard 

Advanced Language Performance. Table 8 contains information on the techniques used 
to assess advanced language performance. Forty-three percent of the states used a state 
developed technique. About one-third of the states used either a local selected or developed 
technique (34%), or a nationally recognized technique (29%). Nine different instruments were 
being used by those states using nationally recognized assessment techniques, with only the Test 
of Adult Basic Education being used by more than one state. Twenty-three percent of the states 
assessing advanced language performance used other assessment techniques such as portfolios 
and course completion. 

Standards approved for use with measures of advanced language performance are 
displayed in Table 9. Of those using state developed assessment techniques, the most frequently 
used standards were percent of students passing (30%) and gain scores (30%). Among states 
using local selected or developed techniques, 84% reported the use of a standard involving a gain 
score. The most frequently reported standards for states using nationally recognized techniques 
were national norm scores (30%) and gain scores (30%). Program/course completion was used 
by 86% of the states reporting the use of other assessment techniques for advanced language 
performance. 
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Table 7 

Standards for Techniques Used to Assess Basic Language Performance (Secondary) (NN40) 



Type of Techniques and Standards f % 

State Developed Techniques 18 
Standards 

% of Students Passing 9 50 

Grade Level/Norm Reference Score 1 5 

Gain Scores 3 17 

Comparison with all Students 3 17 

Local Program Determined 1 5 

To be Determined at State Level 1 5 

Local Selected or Developed Techniques 13 

Standards 

Grade Level/Norm Reference Score 1 i 

Gain Scores 9 69 

Local Program Determined 3 23 

Nationally Recognized Techniques 13 

Standards 

National Norm Score 4 30 

Gain Scores 5 39 

Comparison with all Students 1 8 

Local Program Determined 2 15 

To be Determined at State Level 1 8 

Other Techniques 4 

Standard 

Course/Program Completion 4 100 
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Table 8 

Techniques Used to Assess Advanced Language Performance (Secondary) (N= 35) 



Assessment Techniques 



% 



State Developed 

State High School Proficiency Exam 
Other State Developed Assessment 

Local Selected or Develop ed 

Nationally Recognized 

Iowa Test of Basic Skills 
Test of Adult Proficiency 
Test of Adult Basic Education 
Comprehensive Test of Basic Skills 
Metropolitan Achievement Test 
Stanford Achievement Test 
Stanford 8 3R Battery 
ACT Work Keys 
VTECS Item Banks 



15 



43 



3 
12 



12 
10 



34 
29 



Other 



8 



23 



Course/Program Completion 
Portfolios 



6 
2 



19 



27 



Table 9 

Standards for Techniques Used to Assess Advanced Language Performance (Secondary) (N=35) 



Type oi Techniques and Standards 


r 
I 




0/ 

vo 


State Developed Techniques 




13 




Standards 








% of Students Passing 


4 






Grade Level/Norm Reference Score 


i 




o 

5 


Gain Scores 


4 




30 


Comparison with all Students 


2 




16 


Local Program Determined 


1 




8 


To be Determined at State Level 


1 




8 


Local Selected or Developed Techniques 




12 




Standards 








Grade Level/Norm Reference Score 


1 




Q 

o 


Gain Scores 


10 




84 


Local Program Determined 


1 




8 


Nationally Recognized Techniques 




10 




Standards 








National Norm Score 


3 




30 


Gain Scores 


3 




ic\ 
M 


Comparison with all Students 


2 




20 


Local Program Determined 


1 




10 


To be Determined at State Level 


1 




10 


Other Techniques 




7 




Standards 






86 


Course/Program Completion 


6 




Not Indicated 


1 




14 



28 



Mathematics 



More states reported the assessment of basic and advanced mathematics performance than 
any other academic skill. Forty-six states reported assessing basic mathematics performance, 
while forty-one states reported assessing advanced mathematics performance (See Table 1). 
Information concerning the techniques and standards used in assessing mathematics performance 
is contained in this section. 

Basic Mathematics Performance. Table 10 displays information regarding the techniques 
used in assessing basic mathematics performance. About one-half (48%) of the states reported 
the use of state developed assessment techniques. Almost one-third (30%) of the states reported 
the use of local selected or developed techniques. Thirty percent of the states reported the use 
of nationally recognized assessment techniques. Of the eight instruments identified, the Iowa Test 
of Basic Skills, Test of Adult Proficiency, Test of Adult Basic Education, Comprehensive Test 
of Basic Skills, and Stanford Achievement Test were all used by more than one state. Nine 
percent of the states used course/program completion as a technique for assessing basic 
mathematics performance. 
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Table 10 

Techniques Used to Assess Basic Mathematics Performance (Secondary) (N=46) 



Assessment Techniques f % 



State Developed 22 48 

State High School Proficiency Exam 10 

Other State Developed Assessment 12 

Local Selected or Developed 14 30 

Nationally Recognized i4 30 

Iowa Test of Basic Skills 3 

Test of Adult Proficiency 2 

Test of Adult Basic Education 2 

Comprehensive Test of Basic Skills 2 

Metropolitan Achievement Test 1 

Stanford Achievement Test 2 

Stanford 8 3R Battery 1 

ACT Work Keys 1 

Other 4 9 



Course/Program Completion 4 



The standards reported to be in use for basic mathematics performance are presented in 
Table 1 1. The percent of students passing was the most frequently used standard for state 
developed techniques, with almost one-half of the states (45%) using this type of standard 
Among the states using local selected or developed assessment techniques, 87% reported the 
using gain scores as a standard Twenty-nine percent of the states using nationally recognized 
techniques used a national norm as the standard, although 36% did not indicate a standard All 
of the states using other techniques for assessing basic mathematics performance reported the use 
of a standard of course or program completion. 
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Table 11 

Standards for Techniques Used to .Assess Basic Mathematics Performance (Secondary) (N=46) 



Type of Techniques and Standards f % 



State Developed Techniques 22 
Standards 

% of Students Passing 10 45 

Grade Level/Norm Reference Score 2 9 

Gain Scores 4 18 

Comparison with all Students 4 18 

Local Program Determined 1 5 

To be Determined at State Level 1 5 

Local Selected or Developed Techniques 16 

Standards 

Grade Level/Norm Reference Score 1 7 

Gain Scores 14 87 

Local Program Determined 1 7 

Nationally Recognized Techniques 14 

Standards 

National Norm Score 4 29 

Gain Scores 3 21 

Comparison with all Students 2 14 

Not Indicated 5 36 

Other Techniques 4 

Standard 

Course/Program Completion 4 100 



Advanced Mathematics Performance. Information pertaining to the techniques used by 
states in assessing advanced mathematics performance is contained in Table 12. Over one-third 
(37%) of the states reported the use of state developed assessment techniques. Similarly, about 
one-third (32%) reported the use of local selected and developed assessment techniques. Twenty- 
seven percent of the states reported using a nationally recognized instrument, with nine different 
instruments identified Of these, only the Test of Adult Basic Education and Stanford 
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Achievement Test were reported by more than one state. Slightly more than one-fourth (27%) of 
the states reported the use of other assessment techniques. Course/program completion was the 
most frequently reported of these other techniques. 

Table 12 

Techniques Used to Assess Advanced Mathematics Performance (Secondary) (N=4 1 ) 



Assessment Techniques 



70 



State Developed 

State High School Proficiency Exam 
Other State Developed Assessment 

Local Selected or Developed 

Nationally Recognized 

Iowa Test of Basic Skills 
Test of Adult Proficiency 
Test of Adult Basic Education 
Comprehensive Test of Basic Skills 
Metropolitan Achievement Test 
Stanford Achievement Test 
Stanford 8 3R Battery 
ACT Work Keys 
VTECS Item Banks 



15 



37 



3 
12 



13 
11 



32 
27 



Other 



11 



27 



Course/Program Completion 
Portfolios 

Grade Advancement 



8 
2 
1 



Table 13 contains information on the standards approved for use in assessing advanced 
mathematics performance. Percent of students passing (31%) and gain scores (25%) were the 
most frequently used standards for those states reporting the use of a state developed assessment 
technique. Gain scores were the most frequently reported standards for both local selected or 
developed (71%) and nationally recognized assessment techniques (55%). Of those states using 
other techniques, 64% used a standard of course/program completion. 
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Table 13 

Standards for Techniques Used to Assess Advanced Mathematics Performance (Secondary) 
(N=4H 



Type of Techniques and Standards f % 



State Developed Techniques 16 
Standards 

% of Students Passing 5 31 

Grade Level/Norm Reference Score 2 13 

Gain Scores 4 25 

Comparison with all Students 3 19 

Local Program Determined 1 6 

To be Determined at State Level 1 6 

Local Selected or Developed Techniques 14 

Standards 

Grade Level/Norm Reference Score 1 7 

Gain Scores 10 71 

Local Program Determined 3 22 

Nationally Recognized Techniques 11 

Standards 

National Norm Score 3 27 

Gain Scores 6 55 

Comparison with all Students 2 18 

Other Techniques 11 

Standards 

Gain or Progress 1 9 

Course/Program Completion 7 64 

Not Indicated 3 27 
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Science 



Nineteen states reported the assessment of basic science performance, while twenty states 
reported the assessment of advanced science performance (See Table 1). Information regarding 
the assessment techniques and standards for use with measures of basic and advanced science 
performance is contained in this section. 

Basic Science Performance. Information regarding the techniques used to assess basic 
science performance is contained in Table 14. Over one-half (58%) of the states reported the use 
of state developed techniques. About one-third (32%) of the states reported using local selected 
or developed techniques. Twenty-one percent of the states reported the use of nationally 
recognized techniques. No one instrument was reported as being used in more than one state. 
Twenty-one percent of the states also reported for the other assessment technique course/program 
completion. 

Table 14 

Techniques Used to Assess Basic Science Performance (Secondary) (N=19) 



Assessment Techniques f % 



State Developed 




11 


58 


State High School Proficiency Exam 
Other State Developed Assessment 


5 
6 






Local Selected or Developed 




6 


32 


Nationally Recognized 




4 


21 


Iowa Test of Basic Skills 
Comprehensive Test of Basic Skills 
Stanford Achievement Test 
Stanford 8 3R Battery 


1 
1 
1 
1 






Other 




4 


21 


Course/Program Completion 


4 







Standards approved for use with measures of basic science performance are depicted in 
Table 15. Percent of students passing was the most frequently used standard (50%) for those 
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states reporting the use of state developed assessment techniques. Eighty-six percent of the states 
reported the use of gain score standards for local selected or developed assessment techniques. 
Fifty percent of the states reporting the use of nationally recognized assessment techniques also 
used a gain score standard All of the states using other techniques reported a course/program 
completion standard 

Table 15 

Standards for Techniques Used to Assess Basic Science Performance (Secondary) (N=19) 



Type of Techniques and Standards f % 



State Developed Techniques 12 
Standards 

% of Students Passing 6 50 

Grade Level/Norm Reference Score 1 8 

Gain Scores 2 17 

Comparison with all Students 2 17 

To be Determined at State Level 1 8 

Local Selected or Developed Techniques 7 

Standards 

Gain Scores 6 86 

Local Program Determined 1 14 

Nationally Recognized Techniques 4 

Standards 

National Norm Score 1 25 

Gain Scores 2 50 

Not Indicated 1 25 

Other Techniques 4 

Standard 

Course/Program Completion 4 100 
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Advanced Science Performance . Table 16 presents information concerning the techniques 
used to assess advanced science performance. About one-half (45%) of the states reported the 
use of a state developed assessment technique. Twenty-seven percent of the states reported the 
use of local selected or developed techniques, while only 14% reported the use of a nationally 
recognized assessment. No one test was reported as being used in more than one state. About 
one-third (32%) of the states reported the use of other assessment techniques. 

Standards used in assessing advanced science performance are presented in Table 17. 
Percent of students passing was the most frequently reported standard (36%) for use with state 
developed assessment techniques, followed by gain scores (27%). Gain scores were the most 
frequently reported standard for use with local selected (86%) and nationally recognized 
techniques (67%). Of those states reporting the use of other assessment techniques, 86% reported 
the use of a standard of course/program completion. 

Table 16 



Techniques Used to Assess Advanced Science Performance (Secondary) (N=22) 



Assessment Techniques f % 



0 



State Developed 10 45 

State High School Proficiency Exam 3 
Other State Developed Assessment 7 

Local Selected or Developed 6 27 

Nationally Recognized 3 14 

Comprehensive Test of Basic Skills 1 
Stanford Achievement Test 1 
VTECS Item Banks 1 

Other 7 32 

Course/Program Completion 6 
Portfolios 1 
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Table 17 

Standards for Techniques Used to Assess Advanced Science Perfor mance (Secondary) (N=22) 



Type of Techniques and Standards 



o, 



5 /o 



State Developed Techniques 11 



Standards 

% of Students Passing 

Grade Level/Norm Reference Score 1 9 



Gain Scores 

Comparison with all Students 
Local Program Determined 



Standards 
Gain Scores 

Local Program Determined 

Nationally Recognized Techniques 

Standards 
Gain Scores 
Not Indicated 

Other Techniques 

Standards 

Course/Program Completion 
Not Indicated 



4 36 



3 27 
1 9 
1 9 



To be Determined at State Level 1 9 

\jocal Selected or Developed Techniques 7 



6 86 
1 14 



2 67 
1 33 



6 86 
1 14 
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Other Academic Performance 

In addition to reading, language, mathematics, and science, other academic skills were 
assessed by a number of states. Among these academic skill areas were social studies, problem 
solving, and -thinking skills. Iwenty-fcur states assessed other basic academic skills, while 
twenty-three states assessed other advanced academic skills (See Table 1). This section contains 
information on the techniques and standards used to assess other academic performance. 

Other Basic Academic Performance Information concerning techniques used to assess 
other basic academic performance is contained in Table 18. Forty-two percent of the states 
reported the use of state developed assessment techniques. Forty-two percent also reported the 
use of local selected or developed techniques, while only eight percent reported the use of 
nationally recognized assessment techniques. None of these states reported using the same test 
Over one-third (38%) of the states reported the use of course/program completion as the other 
assessment technique. 

Table 18 

Techniques Used to Asses s Other Basic Academic Performance fSeconrfaty) (N=24) 



Assessment Techniques f 0 



State Developed 10 *y 

State High School Proficiency Exam 6 
Other State Developed Assessment 4 

Local Selected or Develop ed 10 42 

Nationally Recognized 2 8 

Comprehensive Test of Basic Skills 1 
ACT Work Keys 1 



Other 

Course/Program Completion 9 



9 38 



30 



38 



Standards for use in assessing other basic academic performance are presented in Table 
19. Of the states using state developed assessment techniques, 55% reported the use of percent 
of students passing, which was the most frequently reported standard Sixty percent of the states 
using local selected or developed assessment techniques reported a gain score standard One-half 
of the states using a nationally recognized assessment technique reported using a national norm 
as a standard, while 78% of those using other assessment techniques reported using a 
course/program completion standard 

Table 19 

Standards for Techniques Used to Assess Other Basic Academic Performance (Secondary) (N=24) 

Type of Techniques and Standards f % 

State Developed Techniques 11 
Standards 

% of Students Passing 6 55 

Grade Level/Norm Reference Score 1 9 

Gain Scores 1 9 

Comparison with all Students 2 18 

To be Determined at State Level 1 9 

Local Selected or Developed Techniques 10 

Standards 

Grade Level/Norm Reference Score 1 10 

Gain Scores 6 60 

Local Program Determined 3 30 

Nationally Recognized Techniques 2 

Standards 

National Norm Score 1 50 

Not Indicated 1 50 

Other Techniques 9 

Standards 

Gain or Progress 1 1 1 

Course/Program Completion 7 78 

Not Indicated 1 1 1 
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Other Advanced Academic Performance, Techniques used to assess other advanced 
academic performance are displayed in Table 20. About one-third (35%) of the states used state 
developed assessment techniques. Twenty-six percent used a local selected or developed 
technique, while only 13% reported the use of a nationally recognized assessment technique. 
However, none of these states reported the use of the same instrument. About one-third (30%) 
of the states reported the use of other assessment techniques, either portfolios or course/program 
completion. 

Table 20 



Techniques Used to Assess Other Advanced Academic Performance (Secondary) (N=23) 



.Assessment Techniques 


f 




% 


oLdiC UCVClUpCU. 




8 


35 


State High School Proficiency Exam 
Other State Developed Assessment 


2 

6 






Local Selected or Developed 




9 


26 


Nationally Recognized 




3 


13 


Comprehensive Test of Basic Skills 
ACT Work Keys 
VTECS Item Banks 


I 

1 

1 






Other 




7 


30 


Course/Program Completion 
Portfolios 


6 
I 







Table 2 1 presents information on the standards used in assessing other advanced academic 
performance. The most frequently reported standard for states using state developed assessment 
techniques was percent of students passing (44%). Sixty percent of the states using a local 
selected or developed technique reported the use of a gain score standard The states using 
standards with nationally recognized techniques were equally divided between a national norm 
a gain score, and a standard not indicated. Fifty-seven percent of states using other assessment 
techniques used a course/program completion standard 
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Table 21 



Standards for Techniques Used to Assess Other Advanced Academic Performance (Secondary) 



(N=23) 


Type of Techniques and Standards 


f 




% 


State Developed Techniques 




9 




ouuiuarus 
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A 




A A 


Grade Level/Norm Reference Score 


1 




11 


Gain Scores 


2 




22 


Comparison with all Students 


1 




11 


To be Determined at State Level 


1 




11 
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in 




Standards 








Grade Level/Norm Reference Score 


1 




10 


Gain Scores 


6 




60 


Local Program Determined 


3 




30 
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National Norm Score 


1 




33 


Gain Scores 


1 




33 


Not Indicated 


1 




33 


Other Techniaues 




7 




Standards 








Gain or Progress 


1 




14 


Course/Program Completion 


4 




57 


Not Indicated 


2 




29 
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Standards for Other Performance 



Standards for other performance were categorized into one of five groups: competency 
attainment, work skill attainment, completion, placement, and high school graduation rates. 
TTiese groups correspond with the requirements for performance measures and standards outlined 
by the Carl D. Perkins Vocational and Applied Technology Education Act of 1990. States were 
not limited to adopting only the measures and standards outlined in the legislation. However, they 
were instructed by law to include at least one of the previously mentioned measures of 
performance (with corresponding standards), regardless of the number or type of additional 
measures approved Since the Perkins Act required states to offer incentives and adjustments for 
service to special populations, this study also examined any additional measures and standards 
dealing with level of service to those individuals and gender equity. 

Information regarding the number of states approving standards for other measures of 
performance is presented in Table 22. The most frequently reported measures of other 
performance were work skill attainment (72%) and any placement (6 1 %). Additional information 
concerning the measures of other performance may be found in A National Study of Approved 
State Systems of Performance Measures and Standards for Vocational Education (McCaslin and 
Headley, 1993). Competency attainment was generally defined by the states as rate of acquisition 
of basic employability skills. Work skill attainment was defined as the rate of attaining 
occupational skill. Course/program completion was defined by the states as the number of 
students fulfilling program requirements. High school graduation rate was defined by the states 
as the rate at which students graduate from school or achieve the GED(General Educational 
Development) equivalent. Any placement referred to rate of placement on any job or continuing 
education after course/program completion. In almost every case, both types of placement also 
included military service. States defined related placement as the rate at which students are 
placed in jobs related to training or in further education after program completion. States 
generally defined service to special populations as a measure of comparison between enrollment 
rates of general population students and special population students. Gender mix referred to the 
percentage of male and female students enrolled in vocational programs. 

In addition to examining the standards approved for use by the states, this study also 
analyzed information concerning the techniques used in assessing performance. Five types of 
techniques used in assessing work skill attainment and competency attainment were identified: 
state developed, local selected or developed, nationally recognized, occupational licensure or 
certification, and other techniques. Statewide techniques were those developed for statewide use 
(e.g., Georgia Competency Checklists and Kansas Occupational Profiles). Local developed or 
selected techniques were those developed or selected by a local education agency. Selection of 
techniques used in this category were totally at the discretion of the local agency. Nationally 
recognized techniques were those developed for use across states and in many cases had national 
norms (e.g., Work Keys by ACT). Licensure and certification included those administered 
through a state, professional or trade organization. Other techniques included, for example, 
completion of vocational course work or portfolios. 
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Table 22 

A Summary of Approved State Standards for Other Measures of Performance ( Secondary') (N=54) 



Approved Standard 

Performance Measure f % 



Competency Attainment 


24 


44 


Work Skill Attainment 


39 


72 


Course/Program Completion 


25 


46 


High School Graduation 


27 . 


50 


Related Placement 


25 


46 


Any Placement 


33 


61 


Service to Special Populations 


28 


52 


Gender Mix 


17 


31 



Competency Attainment 

Twenty-four states reported assessing competency attainment of secondary vocational 
education students (See Table 22). Standards in this area included those related to the assessment 
of basic employability or transferable skills. Information concerning the techniques and standards 
used in the assessment of competency attainment is contained in this section. 

Techniques. Table 23 presents information related to the techniques used in assessing 
competency attainment. Twenty-nine percent of the states reported the use of state developed 
techniques. Two-thirds of the states (67%) reported using local selected or developed techniques. 
About one-fifth (21%) of the states reported using nationally recognized techniques, with four 
different instruments being identified Only eight percent of the states reported the use of 
licensure or certification, while almost one-third (29%) reported using other techniques. 
Completion of vocational wourse/program was the most frequendy reported other technique for 
assessing competency attainment. 
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Table 23 

Techniques Used to Assess Competency Attainment (Secondary) (N=24) 



Assessment Technique f % 

State Developed 7 29 

Local Selected or Developed 16 67 

Nationally Recognized 5 21 

Jobs for American Graduates 1 

Workplace Readiness Assessment 2 

Work Keys ACT 1 

Youthwork 1 

Occupational Licensure or Certification 2 8 

Other 7 29 

Vocational Course/Program Completion 6 
Portfolios 1 



Standards. Information regarding the standards used in the assessment of competency 
attainment are presented in Table 24. Of those states reporting the use of state developed 
assessment techniques, the most frequently used standard was percentage of students attaining 
basic competency (43%). Sixty-nine percent of the states utilizing local selected or developed 
assessment and 80% of those using nationally recognized techniques reported the use of 
percentage of students achieving basic competency as a standard Both of those states utilizing 
licensure and certification as techniques for assessing competency attainment reported attainment 
of the license or certificate as the standard Of the states using other assessment techniques, over 
one-half (57%) did not indicate a standard, while all those that did indicate a standard (43%) 
reported course/program completion as the standard 
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Table 24 

Standards for Techniques Used in Assessing Competency Attainment of Vocational Students 
(Secondary) (N=24) 



Type of Technique f % 



State Developed 7 
Standards 

% of Students Attaining Basic Competency 3 43 

To be Determined at State Level 2 28 

Not Indicated 2 28 

Local Selected or Developed 16 

Standards 

% of Students Attaining Basic Competency 11 69 

Gain Scores 1 6 

Local Program Determined 3 19 

Not Indicated 1 6 

Nationally Recognized t 5 

Standards 

% of Students Attaining Basic Competency 4 80 

Not Indicated 1 20 

Licensure or Certification 2 

Standard 

Attainment of License or Certification 2 100 

Other 7 



Standards 

Course/Program Completion 3 43 

Not Indicated 4 57 
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Work Skill Attainment 



Thirty-nine states reported assessing work skill attainment of secondary vocational 
education students (See Table 22). Information concerning the techniques and standards used in 
the assessment of competency attainment is contained in this section. 

Techniques. Table 25 presents information related to the techniques used in assessing 
work skill attainment. Twenty-eight percent of the states reported the use of state developed 
techniques. Over two-thirds of the states (69%) reported using local selected or developed 
techniques. Only five percent of the states (N=2) reported using nationally recognized 
techniques, with both states reporting different instruments being used Thirteen percent of the 
states reported the use of licensure or certification, while slightly more than one-fifth (21%) 
reported using other techniques. Completion of vocational course/program was the most 
frequently reported other technique for assessing competency attainment. 

Table 25 



Techniques Used to Assess Work Skill Attainment (Secondary) (N=39) 



Assessment Technique 


f 




% 


State Developed 




n 


28 


Test 

Checklist 


6 
5 






Local Selected or Developed 




27 


69 


Nationallv Recognized 




2 


5 


VTECS Materials 
NOCTI Exams 


1 

1 






Occupational Licensure or Certification 




5 


13 


Other 




8 


21 


Vocational Course/Program Completion 
Portfolios 

GPA in Vocational Courses 


5 
2 

i 







ERIC 



38 

46 



Standards. Information regarding the standards used in the assessment of work skill 
attainment is presented in Table 26. Of those states reporting the use of state developed 
assessment techniques, the most frequently used standards were percentage of students attaining 
basic competency (27%) and gain scores (27%). Eighty-five percent of the states utilizing a local 
selected or developed assessment reported the use of percentage of students achieving basic 
competency as a standard Of the two states reporting the use of nationally recognized 
techniques, one utilized a national norm as a standard, and one used a standard of gain scores. 
Eighty percent of those states utilizing licensure and certification reported attainment of the 
license or certificate as the standard Of the states using other assessment techniques, one-half 
(50%) did not indicate a standard, those states that did indicate a standard (50%) reported 
course/program completion as the standard 
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Table 26 

Standards for Techniques Used in Assessing Work Skill Attainment (Secondary) (N=39) 



Type of Technique f % 



State Developed 11 
Standards 

% of Students Attaining Passing Score 3 27 

Gain Scores 3 27 

To be Determined at State Level 2 22 

Not Indicated 2 22 

Local Selected or Developed 27 

Standards 

% of Students Attaining Passing Score 23 85 

Gain Scores 2 7 

Local Program Determined 1 4 

To be Determined at State Level 1 4 

Nationally Recognized 2 

Standards 

National Norm 1 50 

Gain Scores 1 50 

Licensure or Certification 5 

Standards 

Attainment of License or Certification 4 80 

Not Indicated 1 20 

Other 8 



Standards 

Course/Program Completion 4 50 

Not Indicated 4 50 



Course/Program Completion 



Twenty-five states reported a standard for course/program completion rates (See Table 22). 
Information concerning course/program completion is contained in this section. 

Table 27 presents information about course/program completion standards. The states 
generally defined completion as completing the requirements of the vocational course/program 
or retention in the program Eight states (32%) reported standards for course/program completion 
of between 80% and 90% 

Table 27 

Standards Used in Assessing Course/Program Completion Rates (Secondary) (N=25) 
Standard f % 



Course^Program Completion and/or Retention Rates 



>90% 


0 


0 


80% - 90% 


8 


32 


70% - 79% 


2 


8 


< 70% 


1 


4 


> Comparable Programs 


1 


4 


Yearly Increase in Rates 


2 


8 


To be Determined 


4 


16 


Yearly Increase in Rates 


1 


4 


Attrition Rate < 10% 


1 


4 


Not Indicated 


5 


20 


Total 


25 


100 




High School Graduation 



Twenty-seven states reported a standard for high school graduation rates (See Table 22). 
Information on the standards used for high school graduation rates are contained in this section. 

Table 28 portrays the data related to standards for high school graduation rates. 
Generally, states defined high school graduation as either graduation or its equivalent (e.g., 
GED). The most frequently reported standard (30%) was graduation rate for vocational students 
to be equal to or greater than the graduation rate for the total student population. Twenty-six 
percent of the states reported a standard of between 90% and 95% for graduation rate. 

Table 28 



Standards Used in Assessing High School Graduation Rates (Secondary) (N=27) 



Standard 


f 


% 


> 95% 


1 


4 


90% - 95% 


7 


26 


80% - 89% 


2 


7 


< 80% 


1 


4 


Rate for Vocational Student 


8 


30 


> Total Student Rate 






Local Established Rate 


1 


4 


To be Determined 


2 


7 


Not Indicated 


5 


18 


Total 


27 


100 



Placement 

This section contains information on the standards approved for assessing placement rates. 
Ninety-two percent of the states reported some type of placement standard (McCaslin and 
Headley, 1993). States defined related placement as the rate at which students are placed in jobs 
related to training or in further education after program completion. Any placement referred to 
rate of placement on any job or continuing education after course/program completion. Forty-six 
percent of the states reported the use of a standard for related placement, while 61% reported the 
use of a standard for any placement (See Table 22). 
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Table 29 presents information concerning standards for placement rates. One-third (33%) 
of the states assessing any placement reported a standard of between 90% and 95% placement 
rate. Over one-fourth (27%) reported a standard of between 80% and 89% The standards 
reported most often for related placement were between 70% and 79% (32% of the states) and 
less than 70%-(24% of the states). 

Table 29 

Standards Used in Assessing Placement Rates (Secondary) (N=33) 



Placement 

Standard Any Related 

f % f % 



90% - 95% 


11 


33 


1 


4 


80% - 89% 


9 


27 


1 


4 


70% - 79% 


3 


9 


8 


32 


< 70% 


2 


6 


6 


24 


Equal to Local Employment Rate 


1 


3 


1 


4 


Annual Improvement in Rate 




9 


2 


8 


Local Program Established 


1 


3 


0 


0 


To be Determined 


3 


9 


2 


8 


Not Indicated 


0 


0 


4 


16 


Total 


33 


99* 


25 


100 



Note:* Column total not equal to 100 due to rounding error. 

Service to Special Populations 

Twenty-eight states specifically reported standards designed to assess service to special 
population students (See Table 22). This section contains information related to the standards 
used in assessing service to special population students. 



Table 30 presents data related to standards used in assessing service to special population 
students. Vocational education special population enrollment being equal to or greater than the 
rate of special population enrollment in the total school or district was the most frequently 
reported standard (71%). Two states defined this standard as a rate greater than 80% of the total 
special population enrollment in the school, while the remainder of these states defined the 
standard as being equal to or greater than the actual special population enrollment in the school 
or district. 

Table 30 

Standards Used in Assessing Service to Special Populations (Secondary) (N=28) 



Standard f % 



Vocational Special Population Enrollment > 
Total Special Population Enrollment 

Vocational Special Population Enrollment 
50% - 75% Total Special Population Enrollment 

Equalized Distribution of Special Populations 
Across Vocational Programs 

Increase in Special Population Enrollment in 
Vocational Programs 

To be Determined 

Not Indicated 



Gender Equity 

Seventeen states specifically reported a standard designed to assess the participation of 
nontraditional gender participation in vocational programs (See Table 22). Information 
concerning these standards is found in Table 3 1 . Thirty-five percent of the states did not indicate 
a standard Vocational gender enrollment rate equaling total gender enrollment rate (24%), and 
yearly increase in gender enrollment rate (24%) were the most frequently reported standards of 
those states indicating a standard 
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1 

1 
1 

4 



71 

4 

4 

4 
4 
14 
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Table 31 

Standards Used in Assessing Gender Participation (Secondary) (N=17) 



Standard 


f 


% 


Vocational Gender Enrollment Rate Equals 
Total Gender Enrollment Rate 


4 


24 


Yearly Increase in Gender Enrollment Rate 


4 


24 


Vocational Gender Enrollment Rate > 25% 


3 


18 


Not Indicated 


6 


35 
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Postsecondary Vocational Education Standards 



Standards of performance for postsecondary vocational education programs was also 
examined in this study. As with secondary vocational education, the major categories required 
by the Carl D. -Perkins Vocational and Applied Technology Education Act Amendments of 1990 
were used in organizing the findings concerning performance standards. These categories 
included academic performance and other performance. 



Academic standards were organized into two groups: basic and advanced Thirty-one percent of 
the states reported the use of one set of standards for both basic and advanced academics 
(McCaslin and Headley, 1993). Information on the number of states with approved standards for 
these areas is presented in Table 32. Standards were reported for use with academic areas 
including reading, language, mathematics, science, and other academic skills. The most 
frequently assessed areas of basic academic performance were mathematics (56%) and language 
(54%). The most frequently assessed areas of advanced academic performance were mathematics 
(48%), language (44%), and other advanced academic performance (44%). 



A Summary of Approved State Standards for Academic Skills f Postsecondary) (N=54) 



Academic Standards 



Table 32 



Academic Skill 



f 



% 



Basic 



Reading 

Language 

Mathematics 

Science 

Other 



26 
29 
30 
8 

27 



48 
54 
56 
15 
50 



Advanced 



Reading 

Language 

Mathematics 

Science 

Other 



16 
24 
26 
8 

24 



30 
44 
48 
15 
44 



46 



54 



In addition to examining the standards which had been approved by the states for use with 
academic performance measures, this study examined the types of techniques used in assessing 
performance. Four major types of techniques were identified: state developed, local selected or 
developed, nationally recognized, and other. Statewide techniques were those developed for 
statewide use" (e.g., New Jersey testing program for collegiate and non-collegiate post secondary 
vocational students). Local developed techniques were those developed or selected by a local 
education agency. In some cases the state suggested some nationally recognized instrument for 
possible use, in other states, the choice of technique was left totally to the local agency. 
Nationally recognized techniques were those developed for use nationwide and in many cases 
used national norms (e.g., Test of Adult Basic Education). The use of these instruments was 
mandated by the state. The category other techniques included, for example, gain or progress 
in a course, or completion of course work. The use of techniques in this category were also 
mandated by the state. 

A number of states reported the use of more than cne assessment technique for each of 
the academic measures; For assessing reading, four states reported the use of more than one 
technique. Four states also reported using more than one assessment technique for mathematics 
and for language. In addition, one state reported the use of more than assessment technique for 
assessing other academic skills. 

Reading 

In the academic area of reading, twenty-six states reported having approved standards for 
basic reading performance and sixteen states reported approved standards for advanced reading 
performance (see Table 32). Information concerning the techniques used to assess reading 
performance and the standards for basic and advanced reading performance is contained in this 
section. 

Basic Reading Performance. Table 33 presents information on the techniques used to 
assess basic reading performance. Only eight percent of the states reported the use of state 
developed assessment techniques. Sixty-nine percent of the states reported the use of local 
selected or developed techniques. Slightly over one-fourth (27%) of the states reported the use 
of nationally recognized techniques. Three different instruments were reported to be in use in 
use; with only the Test of Adult Basic Education being used by mare than one state. Thirty-eight 
percent of the states reported using other techniques, such as course/program completion, grade 
point average (GPA) and graduate equivalency diploma (GED) criteria 
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Table 33 



Techniques Used to Assess Basic Reading Performance f Postsecondary) (N=26) 



Assessment Techniques 


f 




% 


State Developed 




2 


8 


Local Selected or Developed 




18 


69 


Nationally Recognized 




7 


27 


Test of Adult Basic Education 
California Achievement Test 
Asset 


5 
1 
1 






Other 




9 


38 


Course/Program Completion 

GED Criteria 

GPA 


6 
1 

2 







Standards approved for use with measures of basic reading performance are displayed in 
Table 34. Of those states reporting the use of state developed assessment techniques, the 
percentage of students receiving a passing score (50%) and a standard determined by the local 
program (50%) were equally reported. Of the states reporting the use of a local selected 
technique, 39%, used gain scores and 33% reported local program determined standards. The 
most frequently reported standard for those using a nationally recognized technique was national 
norm scores (57%). Almost one-half (44%) of the states which employed other techniques for 
assessing basic reading performance reported a standard of course/program completion. 

Advanced Reading Performance. Table 35 presents information on the techniques used 
to assess advanced reading performance. Six percent of the states measuring advanced reading 
performance reported the use of state developed assessment techniques. Three-fourths (75%) of 
the states reported using local selected or developed techniques. One-fourth (25%) of the states 
assessing advanced reading performance reported the use of nationally recognized techniques. 
Two instruments were reported to be in use; with the Test of Adult Basic Education being used 
by more than one state. About one-third of the states (31%) reported the use of other 
techniques, such as course/program completion and GPA. 
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Table 34 

Standards for Techniques Used to Assess Basic Reading Performance ( Postsecondar/) (N=26) 



Type of Techniques and Standards f % 



State Developed Techniques 2 
Standards 

% of Students Passing 1 50 

Local Program Determined 1 50 

Local S el ected or Developed Techniques 18 

Standards 

Grade Level/Norm Reference Score 4 22 

Gain Scores 7 39 

Local Program Determined 6 33 

To be Determined 1 6 

Nationally Recognized Techniques 7 

Standards 

National Norm Score 4 57 

Gain Scores 2 29 

Not Indicated 1 14 

Other Techniques 9 

Standards 

Gain or Progress 1 1 1 

Course/Program Completion 4 44 

Not Indicated 2 22 

GPA 2 22 
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Table 35 



Techni ques Used to Assess Advanced Reading Performance (Postsecondarv^ (N=16) 



■Assessment Techniques 


f 




% 


State Developed 




1 


6 


Local Selected or Developed 




12 


75 


Nationally Recognized 




4 


25 


Test of Adult Basic Education 
Asset 


3 
I 






Other 




5 


31 


Course/Program Completion 
GPA 


4 
1 







Standards approved for use with measures of advanced reading performance are displayed 
in Table 36. The one state reporting the use of a state developed assessment technique used a 
gain score as a standard Of the states reporting the use of a local selected technique 50% used 
standards determined by the local program The most frequently reported standard for those using 
a nationally recognized technique was a national norm score (50%). Eighty percent of the states 
which employed other techniques for assessing advanced reading performance reported using a 
course/program completion standard 
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Table 36 

Standards for Techniques Used to Assess Advanced Reading. Performance (Postsecondarv) (N= 1 6) 



Type of Techniques and Standards f % 



State Developed Techniques 1 
Standards 

Gain Scores 1 100 

Local Selected or Developed Techniques 12 
Standards 

Grade Level/Norm Reference Score 2 17 

Gain Scores 4 33 

Local Program Determined 6 50 

Nationally Recognized Techniques 4 

Standards 

National Norm Score 2 50 

Gain Scores 1 25 

Not Indicated 1 25 

Other Techniques 5 

Standards 

Course/Program Completion 4 80 

GPA 1 20 



Language 

Twenty-nine states reported assessing basic language performance, while twenty-four 
states reported assessing advanced academic performance (see Table 32). This section contains 
information regarding the techniques and standards used to assess language performance, 

Basic Language Performance. Table 37 presents information on the techniques used to 
assess basic language performance. Ten percent of the states measuring basic language 
performance reported the use of state developed assessment techniques. Fifty-five percent of the 
states reported the use of local selected or developed techniques. About one-fifth of the states 
(17%) reported the use of nationally recognized techniques. Two different instruments were 
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reported to be in use and the Test of .Adult Basic Education was being used in more than one 
state. Forty-one percent of the states measuring basic language performance reported using other 
assessment techniques such as course/program completion. 

Table 37 - 



Techniques Used to Assess Basic Laneuaee 


Performance (Postsecondarv) (N=29) 




Assessment Techniques 


f 




% 


State Developed 




3 


10 


Local Selected or Developed 




16 


55 


Nationallv Recognized 




5 


17 


Test of Adult Basic Education 
Asset 


4 
I 






Other 




12 


41 


Course/Program Completion 
GED Criteria 
Institutional Requirements 
GPA 


9 
1 
1 
1 







Standards approved for use with measures of basic language performance are displayed 
in Table 38. Of those states reporting the use of state developed techniques, the percentage of 
students receiving a passing score was the most frequently reported (67%) standard Gain scores 
were the most frequently used standard for local selected or developed techniques with 50% of 
the states measuring basic language performance reporting their use. National norm scores were 
the most frequently reported (60%) standard for nationally recognized assessment techniques. 
Of the states reporting other techniques for assessing basic language performance, 67% used a 
course/program completion standard 



52 

till 



Table 38 

Standards for Techniques Used to Assess Basic Language Performance (PostsecondarV) (N=29) 



Type of Techniques and Standards f % 



State Developed Techniques 3 
Standards 

% of Students Passing 2 67 

Comparison with all Students I 33 

Local Selected or Developed Techniques 16 

Standards 

Grade Level/Norm Reference Score 4 25 

Gain Scores 8 50 

Local Program Determined 4 25 

Nationally Recognized Techniques 5 

Standards 

National Norm Score 3 60 

Gain Scores 1 20 

Not Indicated I 20 

Other Techniques 12 

Standards 

Course/Program Completion 8 67 

Not Indicated 2 17 

Comparison with all Student 1 8 

GPA 1 8 
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Advanced Language Performance. Table 39 contains information on the techniques used 
to assess advanced language performance. Thirteen percent of the states used a state developed 
technique. One-half (50%) of the states used a local selected or developed technique, while only 
17% used a nationally recognized assessment technique. Two different instruments were being 
used by those -States using nationally recognized assessment techniques, with only the Test of 
Adult Basic Education being used by more than one state. Fifty percent of the states used other 
assessment techniques such as course/program completion. 

Standards approved for use with measures of advanced language performance are 
displayed in Table 40. Of those three states using state developed assessment techniques, an 
equal number reported using standards o£ percent of students passing, gain scores, and a 
comparison with all students. Among states using local selected or developed techniques, 42% 
reported the use of a standard involving a gain score. The most frequently reported standard in 
use in states using nationally recognized techniques was a national norm scores (50%). 
Course/program completion was used by 83% of the states reporting the use of other assessment 
techniques. 

Table 39 

Techniques Used to Assess Advanced Language Performance (Postsecondarv) (N=24) 



Assessment Techniques 


f 




% 


State Developed 




3 


13 


Local Selected or Developed 




12 


50 


Nationallv Recognized 




4 


17 


Test of Adult Basic Education 
Asset 


3 
1 






Other 




12 


50 


Course/Program Completion 
Institutional Requirements 
GPA 


10 
1 
1 
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Table 40 

Standards for Techniques Used to Assess Advanced Language Performance (Postsecondary) 
(N=24) 



Type of Techniques and Standards f % 



State Developed Techniques 3 
Standards 

% of Students Passing 1 33 

Gain Scores 1 33 

Comparison with all Students 1 33 

Local Selected or Developed Techniques 12 

Standards 

Grade Level/Norm Reference Score 2 17 

Gain Scores 5 42 

Local Program Determined 4 33 

To be Determined 1 8 

Nationally Recognized Techniques 4 

Standards 

National Norm Score 2 50 

Gain Scores 1 25 

Not Indicated 1 25 

Other Techniques 12 

Standards 

Course/Program Completion 10 83 

Comparison with all Students 1 8 

GPA 1 8 
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Mathematics 



More states reported the assessment of basic and advanced mathematics performance than 
any other academic skill. Thirty states reported assessing basic mathematics performance, while 
twenty-six reported assessing advanced mathematics performance (See Table 32). Information 
concerning the techniques and standards used in assessing mathematics performance is contained 
in this section. 

Basic Mathematics Performance. Table 41 displays information regarding the techniques 
used in assessing basic mathematics performance. Only seven percent of the states reported the 
use of state developed assessment techniques. Conversely, 60% of the states reported the use of 
local selected or developed techniques. Seventeen percent of the states reported the use of 
nationally recognized assessment techniques. Of the two instruments identified, the Test of Adult 
Basic Education was the only one used by more than one state. Forty-seven percent of the states 
used other techniques, with course/program completion being used in the majority of these states. 

Table 41 

Techniques Used to Assess Basic Mathematics Performance (Tostsecondarv) (N=30) 



Assessment Techniques 


f 




% 


State Developed 




2 


7 


Local Selected or Developed 




18 


60 


Nationally Recognized 




5 


17 


Test of Adult Basic Education 
Asset 


4 
1 






Other 




14 


47 


Course/Program Completion 
GED Criteria 
Institutional Requirements 
GPA 


10 
1 
1 

2 
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The standards reported to be in use for basic mathematics performance are presented in 
Table 42. The percent of students passing and a to be determined standard were the most 
frequently used standards for state developed techniques (each 50%). Among the states using 
local selected or developed assessment techniques, 42% reported using gain scores as a standard 
Forty percent of the states using nationally recognized techniques used a national norm as the 
most frequently reported standard Sixty-four percent of the states using other techniques 
reported the use of a standard of course/program completion. 

Table 42 

Standards for Techniques Used to Assess Basic Mathematics Performance (Postsecondary) (N=30) 



Type of Techniques and Standards f % 



State Developed Techniques 2 
Standards 

% of Students Passing 1 50 

To be Determined at State Level 1 50 

Local Selected or Developed Techniques 19 

Standards 

Grade Level/Norm Reference Score 4 21 

Gain Scores 8 42 

Local Program Determined 6 32 

To be Determined 1 5 

Nationally Recognized Techniques 5 

Standards 

National Norm Score 2 40 

Gain Scores 1 20 

To be Determined at State Level 2 40 

Other Techniques 14 

Standards 

Course/Program Completion 9 64 

Not Indicated 2 14 

Comparison with all Students I 8 

GPA 2 14 
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Advanced Mathematics Performance. Information pertaining to the techniques used by 
states in assessing advanced mathematics performance is contained in Table 43. Only eight 
percent of the states reported the use of state developed assessment techniques. Fifty percent 
reported the use of local selected and developed assessment techniques. Fifteen percent of the 
states reported using a nationally recognized instrument, with just two different instruments 
identified Of these, only the Test of Adult Basic Education was reported to be in use by more 
than one state. More than one-third (38%) of the states reported the use of an other assessment 
technique. Course/program completion was the most frequently reported of the other techniques. 

Table 43 



Techniques Used to Assess Advanced Mathematics Performance (Postsecondary) (N=26) 



Assessment Techniques 


f 




% 


State Developed 




2 


8 


Local Selected or Developed 




13 


50 


Nationallv Recognized 




4 


15 


Test of Adult Basic Education 
Asset 


3 
1 






Other 




10 


38 


Course/Program Completion 
Institutional Requirements 
GPA 


8 
1 
1 







Table 44 contains information on the standards approved for use in assessing advanced 
mathematics performance. Percent of students passing and gain scores were both used as 
standards in one-half (50%) of those states reporting the use of a state developed assessment 
technique. Gain scores (38%) and standards determined by the local programs (38%) were the 
most frequently reported standards for local selected or developed assessment techniques. Of 
those states using a nationally recognized technique for assessing advanced mathematics 
performance, 50% used a national norm score standard. Seventy percent of those states using 
other techniques reported the use of course/program completion as a standard. 
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Table 44 



Standards for Techniques Used to Assess Advanced Mathematics Performance (Postsecondary) 



(N=26) 


1 yjJC KJl ICClUUV^UCb allU oUuluiuUo 


f 
i 




/O 


State Developed Techniques 




2 




Standards 








yo ot students Passing 


i 
l 






Gain Scores 


i 




50 


Local Selected or Developed Techniques 




13 




Standards 








Grade Level/Norm Reference Score 


2 




15 


Gain Scores 


C 

J 




DO 


Local Program Determined 


5 




38 


1 (J UC i^CLCIIIUIlCU 


i 
i 




O 


Nationally Recognized Techniques 




4 




Standards 








National Norm Score 


2 




50 


Gain Scores 


1 




25 


To be Determined at State Level 


1 




25 


Other Techniques 




10 




Standards 








Course/Program Completion 


7 




70 


Not Indicated 


1 




10 


Comparison with all Students 


1 




10 


GPA 


1 




10 
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Science 



Eight states reported the assessment of both basic and advanced science performance (See 
Table 32). Information regarding the assessment techniques and standards for use with measures 
of basic and advanced science performance is contained in this section. 

Basic Science Performance. Information regarding the techniques used to assess basic 
science performance is displayed in Table 45. Eighty-seven percent of the states reported the use 
of local selected a developed techniques. Thirteen percent of the states reported the use of an 
other assessment technique. No state reported using a nationally recognized or state developed 
assessment technique for assessing basic science performance. 

Standards approved for use with measures of basic science performance are depicted in 
Table 46. Gain scores were the most frequently reported standard (57%) for those states 
reporting the use of local developed or selected assessment techniques. The state using other 
techniques for assessing basic science performance reported a comparison with all students 
standard 



Table 45 



Techniques Used to Assess Basic Science Performance (Postsecondarv) (N=8) 



Assessment Techniques 



f 



% 



State Developed 



0 



0 



Local Selected or Developed 



7 



87 



Nationally Recognized 



0 



0 



Other 



13 



Institutional Requirements 



G3 



Table 46 

Standards for Techniques Used to .Assess Basic Science Performance (Postsecondarv) (N=8) 



Type of Techniques and Standards f % 







0 




Local Selected or Developed Techniaues 




7 




Standards 








Gain Scores 


4 




57 


Local Program Determined 


2 




29 


To be Deteimined 


1 




14 


Nationally Recognized Techniaues 




0 




Other Techniques 




1 




Standard 








Comparison with all students 


1 




100 



Advanced Science Performance. Table 47 presents information concerning the techniques 
used to assess advanced science performance. Eighty-seven percent of the states reported the use 
of a local developed or selected assessment technique. Thirteen percent of the states reported the 
use of other assessment techniques. No state reported the use of state developed or nationally 
recognized assessment techniques for advanced science performance. 



Standards used in assessing advanced science performance are presented in Table 48. 
Gain scores (43%) and standards determined by the local program (43%) were the most 
frequently reported standards used for local selected or developed assessment techniques. The 
one state reporting the use of other assessment techniques utilized a standard of comparison with 
all students. 
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Table 47 

Techniques Used to Assess Advanced Science Performance (Postsecondary) (N=8) 



Assessment Techniques 



State Developed 

Local Selected or Developed 

Nationally Recognized 

Other 

Institutional Requirements 



% 



0 
7 
0 
1 



0 
87 

0 
13 



Table 48 

Standards for Techniques Used to Assess Advanced Science Performance (Postsecondary) (N=8) 



Type of Techniques and Standards 


f 




% 


State Developed Techniques 




0 




Local Selected or Developed Techniques 




7 




Standards 
Gain Scores 

Local Program Determined 
To be Determined 


3 
3 
1 




43 
43 
14 


Nationally Recognized Techniques 




0 




Other Techniques 




1 




Standard 

Comparison with all Students 


1 




100 
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Other Academic Performance 



In addition to reading, language, mathematics, and science, other academic skills were 
assessed by a number of states. Among these academics were social studies, problem solving, 
and thinking skills. Twenty-seven states assessed other basic academic skills, while twenty-four 
states assessed other advanced academic skills (See Tabic 32). This section contains information 
on the techniques and standards used to assess other academic performance. 

Other Basic Academic Performance. Information concerning techniques used to assess 
other basic academic performance is contained in Table 49. Only four percent of the states 
reported the use of state developed assessment techniques. Fifty-six percent reported the use of 
local selected or developed techniques, while no state reported the use of a nationally recognized 
assessment technique. Over one-third (40%) of the states reported the use of other assessment 
techniques, the most frequently reported of these being course/program completion. 

Table 49 

Techniques Used to Assess Other Basic Academic Performance (Postsecondary) (N=27) 

Assessment Techniques f % 

State Developed 1 4 

Local Selected or Developed 15 56 

Nationally Recognized 0 0 

Other 11 40 

Course/Program Completion 6 

GED Criteria 1 

GPA 3 

Student Opinion Survey 1 
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Other Advanced Academic Performance. Techniques used to assess other advanced 
academic performance are displayed in Table 51. Only four percent of the states used state 
developed assessment techniques. Fifty percent used a local selected or developed technique, 
while no state reported the use of a nationally recognized assessment technique. About one-half 
(46%) of the states reported the use of other assessment techniques, for example, course/program 
completion or GPA. 

Table 50 

Standards for Techniques Used to Assess Other Basic Academic Performance (Postsecondarv) 
(N=27) 



Type of Techniques and Standards f % 



State Developed Techniques 1 
Standards 

% of Students Passing 1 1 00 

Local Selected or Developed Techniques 15 
Standards 

Grade Level/Norm Reference Score 3 20 

Gain Scores 5 33 

Local Program Determined 5 33 

Not Indicated 2 14 

Nationally Recognized Techniques 0 

Other Techniques H 

Standards 

Course/Program Completion 5 45 

Not Indicated 3 27 

GPA 3 27 
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Table 51 

Techniques Used to Assess Other Advanced Academic Performance (Postsecondary) (N=24) 



Assessment Techniques f % 

State Developed 1 4 

Local Selected or Developed 12 50 

Nationally Recognized 0 0 

Other 11 46 

Course/Program Completion 6 
GPA 4 
Student Opinion Survey 1 
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Table 52 presents information on the standards used in assessing other advanced academic 
performance. A gain score standard was used by the state reporting a state developed technique 
for assessing advanced other academic performance. Fifty percent of the states using a local 
selected or developed technique reported the use of a standard determined by the local program. 
Forty-five percent of states using other assessment techniques used a course/program completion 
standard, while 37% of states utilized GPA. 

Table 52 

Standards for Techniques Used to Assess Other Advanced Academic Performance 



(Postsecondarv) fh-24") 


Type of Techniques and Standards 


f 






State Developed Techniques 




1 




Standards 








Gain Scores 


1 




100 


Local Selected or Developed Techniques 




12 




Standards 








Grade Level/Norm Reference Score 


2 




17 


Gain Scores 


4 




33 


Local Program Determined 


6 




50 


Nationally Recognized Techniques 




0 




Other Techniques 




11 




Standards 








Course/Program Completion 


5 




45 


Not Indicated 


2 




18 


GPA 


4 




37 
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Standards for Other Performance 



Standards for other performance were classified in one of four groups: competency 
attainment, work skill attainment, course/program completion, and placement These groups 
corresponded with the requirements for performance measures and standards outlined by the Carl 
D. Perkins Vocational and Applied Technology Education Act of 1990. States were not limited 
to adopting only the measures and standards outlined in the legislation. However, they were 
instructed by law to include at least one of the previously mentioned measures of performance 
(with corresponding standards), regardless of the number or type of additional measures approved 
Since the Perkins Act required states to offer incentives and adjustments for service to special 
populations, this study also examined any additional measures and standards dealing with level 
of service to those individuals and gender equity. 

Information regarding the number of states approving standards for other measures of 
performance is presented in Table 53. The most frequently reported measures of other 
performance were program completion (70%) and work skill attainment (63%). Information 
concerning the measures of other performance may be found in A National Study of Approved 
State Systems of Performance Measures and Standards for Vocational Education (McCaslin and 
Headley, 1993). 

Competency attainment was generally defined by the states as the rate of acquisition of 
basic employability skills. Work skill attainment was defined as the rate of attaining occupational 
skill. Course/program completion was defined by the states as the number of students fulfilling 
program requirements. Any placement referred to rate of placement on any job or continuing 
education after course/program completion. States defined related placement as the rate at which 
students are placed in jobs related to training or in further education after course/program 
completion. In almost all states, both types of placement included entry into the military. States 
generally defined service to special populations as a measure of comparison between enrollment 
rates of general population students and special population students. Gender mix referred to the 
percentage of male and female students enrolled in vocational education programs. 

In addition to examining the standards approved for use by the states, this study also 
analyzed information concerning the techniques used in assessing performance. Five types of 
techniques used in assessing work skill attainment and competency attainment were identified: 
state developed, local selected or developed, nationally recognized, occupational licensure or 
certification, and other techniques. State developed techniques were those approved for statewide 
use (e.g., Kansas Occupational Profiles). Local developed or selected techniques were those 
developed or selected by a local education agency. Selection of techniques used in this category 
were totally at the discretion of the local agency. Nationally recognized techniques were those 
developed for use across states and in many cases were nationally normed (e.g., Work Keys by 
ACT). Licensure and certification included those administered through a state, professional or 
trade organization. Other techniques included, for example, completion of vocational course work 
or portfolios. 
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Table 53 



A Summary of Approved State Standards for Other Measures of Performance (Postsecondary) 
(N=54) 





Approved 


Standards 


Performance Measure 








f 


% 


Competency Attainment 


18 


33 


Work Skill Attainment 


34 


63 


Course/Program Completion 


38 


70 


Related Placement 


26 


48 


Any Placement 


30 


56 


Service to Special Populations 


30 


56 


Gender Mix 


17 


31 



Competency Attainment 

Eighteen states reported the assessment of competency attainment (See Table 53). 
Information regarding the techniques and standards used in assessing competency attainment is 
contained in this section. 

Techniques. Table 54 presents information related to the techniques used in assessing 
competency attainment. Eleven percent of the states reported the use of state developed 
techniques. Two-thirds of the states (67%) reported using local selected or developed techniques. 
About one-fifth (21%) of the states reported using nationally recognized techniques, with four 
different instruments being identified The Workplace Readiness Assessment Instrument was the 
only test being used in more than one state. Only eight percent of the states reported the use of 
licensure or certification, while almost one-third (29%) reported using other techniques. 
Completion of vocational course/program work was the most frequently reported other technique 
for assessing competency attainment. 
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Table 54 

Techniques Used to Assess Competency Attainment (Postsecondary) (N=18) 



Assessment Technique f % 

State Developed 2 1 1 

Local Selected or Developed 12 67 

Nationally Recognized 5 21 

Jobs for American Graduates 1 

Workplace Readiness Assessment 2 

Work Keys ACT 1 

Youthwork 1 

Occupational Licensure or Certification 0 8 

Other 7 29 

Vocational Course/Program Completion 6 
Portfolios 1 



Standards, Information regarding the standards used in the assessment of competency 
attainment are presented in Table 55. Of those states reporting the use of state developed 
assessment techniques, the most frequently used standard was percentage of students attaining 
basic competency (50%). Fifty percent of the state did not indicate a standard Fifty percent of 
the states utilizing local selected or developed assessment and 60% of those using nationally 
recognized techniques reported the use of percentage of students achieving basic competency as 
a standard Of the states using other assessment techniques, almost three-fourths (71%) reported 
course/program completion as the standard 

Work Skill Attainment 

Thirty-four states reported assessing work skill attainment of secondary vocational educa- 
tion students (See Table 53). Information concerning the techniques and standards used in the 
assessment of competency attainment is contained in this section. 



69 ( 7 



Techniques. Table 56 presents information related to the techniques used in assessing 
work skill attainment. Twelve percent of the states reported the use of state developed tech- 
niques. Over one-half of the states (56%) reported using local selected or developed techniques. 
Only one of the states reported using nationally recognized techniques, with NOCTI Exams being 
the only instrument reported Twenty-one percent of the states reported the use of licensure or 
certification, while about one-fourth (24%) reported using other techniques. Completion of voca- 
tional course work and GPA in vocational course work were the most frequently reported other 
techniques. 

Table 55 

Standards for Techniques Used in Assessing Competency Attainment (Postsecondary) (N=18) 



Type of Technique f % 



State Developed 2 
Standards 

% of Students Attaining Basic Competency 1 50 

Not Indicated 1 50 

Local Selected or Developed 12 

Standards 

% of Students Attaining Basic Competency 6 50 

Gain Scores 1 8 

Local Program Determined 1 8 

To be Determined at State Level 1 8 

Not Indicated 3 25 

Nationally Recognized 5 

Standards 

% of Students Attaining Basic Competency 3 60 

Not Indicated 2 40 

Licensure or Certification 0 

Other 7 



Standards 

Course/Program Completion 
Not Indicated 
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Table 56 



Techniques Used to Assess Work Skill Attainment fPostsecondarv) (N=34) 



Assessment Technique 


f 




% 


State Developed 




4 


12 


Test 

Checklist 


3 
1 






Local Selected or Developed 




19 


56 


Nationally Recognized 




1 


3 


NOCTI Exams 


1 






Occupational Licensure or Certification 




7 


21 


Other 




8 


24 ' 


Vocational Course/Program Completion 
GPA in Vocational Courses 
Degree 

Student Opinion Survey 


3 
3 
1 
1 







Standards. Information regarding the standards used in the assessment of work skill at- 
tainment are presented in Table 57. Of those four states reporting the use of state developed 
assessment techniques, an equal number (25%) reported the use of percent of students achieving 
basic competency, gain scores, a yet to be determined standard, and a standard that was not indi- 
cated. Seventy-nine percent of the states utilizing a local selected or developed assessment re- 
ported the use of percentage of students achieving a passing score as a standard The state re- 
porting the use of a nationally recognized technique reported the utilization of a national norm 
as a standard Fifty-seven percent of those states utilizing licensure and certification reported 
attainment of the license or certificate as the standard Of the states using other assessment tech- 
niques, one-half (50%) did not indicate a standard and 37% reported course/program completion 
as the standard 
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Table 57 

Standards for Techniques Used in Assessing Work Skill Attainment (Tostsecondarv) (N=34) 



Type of Technique f % 



State Developed 4 
Standards 

% of Students Attaining Passing Score 1 25 

Gain Scores 1 25 

To be Determined at State Level 1 25 

Not Indicated 1 25 

Local Selected or Developed 19 

Standards 

% of Students Attaining Passing Score 15 79 

Gain Scores 1 5 

Not Indicated 3 16 

Nationally Recognized 1 

Standard 

National Norm 1 100 

Licensure or Certification 7 
Standards 

Attainment of License or Certification 4 57 

Not Indicated 3 43 

Other 8 



Standards 

Course/Program Completion 3 37 

Loca 1 Determined 1 13 

Not Indicated 4 50 
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Thirty-eight states reported a standard for program completion rates (See Table 53). In- 
formation concerning program completion is contained in this section. 

Table 58 presents information about program completion standards. The states generally 
defined completion as completing the requirements of the vocational program or retention in the 
program Seventeen of the states did not indicate a standard The most frequently reported stan- 
dards were between 80% and 90% rate (13%), and less than 70% rate (13%). 

Table 58 

Standards Used in Assessing Student Completion of Vocational Programs (Postsecondary) (N=38) 



Standard f % 

Program Completion and/or Retention Rates 

> 90% 0 0 
80% - 90% 5 13 
70% - 79% 3 8 
< 70% 5 13 
Yearly Increase in Rate 1 3 
To be Determined 4 1 1 

Comparison to Institutional Completion Rate 

> Institutional Completion Rate 2 5 
Attrition Rate < 10% 1 3 
Not Indicated 17 44 
Total 38 100 
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Placement 



This section contains information on the standards approved for assessing placement rates. 
Eighty-eight percent of the states reported some type of placement standard (McCaslin and 
Headley, 1993). States defined related placement as the rate at which students are placed in jobs 
related to training or in further education after program completion. Any placement referred to 
rate of placement on any job or continuing education after program completion. Twenty-six of 
the states (48%) reported the use of a standard for related placement, while thirty states (56%) 
reported the use of a standard for any placement (See Table 53). 

Table 59 presents information concerning standards for placement rates. One-fifth (20%) 
of the states assessing any placement reported a standard of a rate of between 80% and 89%. 
Almost one-fifth of these states (17%) reported a standard of over 90%. The most frequently 
reported standard for related placement were 80% - 89% (31% of the states) and 70% - 79% 
(27% of the states). 

Table 59 

Standards Used in Assessing Placement Rates (Postsecondary) (N=30) 



Any Placement Related Placement 

Standard f % f % 



90% - 95% 


5 


17 


1 


4 


80% - 89% 


6 


20 


8 


31 


70% - 79% 


1 


3 


7 


27 


< 70% 


0 


0 


3 


11 


Equal to Local Employment Rate 


1 


3 


0 


0 


Annual Improvement in Rate 


3 


10 


1 


4 


Local Program Established 


1 


3 


0 


0 


To be Determined 


3 


10 


1 


4 


Not Indicated 


10 


33 


5 


19 


Total 


30 


99* 


26 


100 



Note:* Column total not equal to 100 due to rounding error. 
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Service to Special Populations 



Thirty states (56%) reported standards designed to assess service to special population 
students (See Table 53). This section contains information related to the standards utilized in 
assessing service to special population students. 

Table 60 presents data related to standards used in assessing service to special population 
students. Vocational education special population enrollment being equal to or greater than the 
rate of special population enrollment in the total school or district was the most frequently report- 
ed standard (40%). Two of theses states defined this standard as a rate greater than 80% of the 
total special population enrollment in the school, while the remainder of the states defined the 
standard as being equal to or greater than the actual special population enrollment in the school 
or district. Forty-seven percent of the states did not indicate a standard 

Table 60 

Standards Used in Assessing Service to Special Populations (Postsecondary) (N=30) 



Standard f % 



Vocational Special Population Enrollment > 
Total Special Population Enrollment 


12 


40 


Percent Special Population Students Enrolled 
Equals Special Population Students Applied 


1 


3 


Increase in Special Population Enrollment in 
Vocational Programs 


1 


3 


To be Determined 


2 


7 


Not Indicated 


14 


47 




Gender Equity 



Seventeen states reported a specific standard designed to assess the participation of 
nontraditional gender participation in vocational programs (See Table 53). Information con- 
cerning these standards is found in Table 61. Fifty-eight percent of the states did not indicate 
a standard Yearly increase in gender enrollment rate (24%) and vocational gender enrollment 
rate equaling total gender enrollment rate (12%) were the most frequently reported standards of 
those states indicating a standard 

Table 61 



Standards Used in Assessing Gender Participation (Postsecondarv) (N=17) 



Standard 


f 


% 


Vocational Gender Enrollment Rate Equals 
Total Gender Enrollment Rate 


2 


12 


Yearly Increase in Gender Enrollment Rate 


4 


24 


Rate of Gender Enrollment Equals Rate of 
Gender Application to Program 


1 ' 


6 


Not Indicated 


10 


58 
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Comparing Standards Across Secondary and Postsecondary 
Vocational Education 

As reported in the previous sections, a number of standards have been developed by states 
for assessing academic and other performance. This section will present findings that permit 
comparisons of the kinds of standards adopted within and across the secondary and postsecondary 
level. Three basic kinds of standards were selected for use by the states. These were criterion- 
based, norm-based and gains-based A number of states did not indicate standards or had not 
developed standards. 

Criterion-based standards were those in which the standard was based on a predetermined 
level of performance, gauged against a known mark, score, or indicator. Examples of this kind 
of standard included a pre-determined percentage of students passing a state developed academic 
assessment test, and a pre-determined percentage of students attaining basic competency. 
Criterion based standards did not require that vocational students be compared to other students 
or that vocational programs be compared with other programs. 

Norm-based standards were those in which the standards were based on comparisons 
between vocational students and other students or those which compared between vocational 
programs. Examples of this kind of standard included those which were used with a nationally 
normed test, such as the Iowa Test of Basic Skills. Other standards of this type included those 
which compared vocational program outcomes with state averages. 

Gains-based standards were those in which the vocational program was assessed based 
on the gain of students or gain in program outcomes. An example of this type of standard is that 
all students show a gain on a state developed assessment from the eleventh to twelfth grade. 
Another type of this standard is that the average score of students on an academic test would 
increase over previous years. 

Academic Standards 

Standards were developed for basic and advanced academic performance at the secondary 
and postsecondary level. Information regarding kinds of standards used in reading, language, 
mathematics, science and other academic performance is presented in this section. 

Reading 

Forty-three states at the secondary level and twenty-six states at the postsecondary level 
assessed basic reading performance (see Table 1 and Table 32). Table 62 portrays information 
concerning the kind of standard used by the states for basic and advanced reading skill. Fifty- 
three standards were developed for secondary basic reading, while thirty-six standards were 
developed at the postsecondary level. For basic reading performance, the most frequently 
reported standards were gains-based TTiirty-two percent of the secondary standards and 28% oi 
the postsecondary standards were gains-based 




Thirty-five states at the secondary level and sixteen states at the postsecondary level 
assessed advanced reading performance (see Table 1 and Table 32). Forty-four standards were 
developed for secondary advanced reading performance. At the postsecondary level, twenty-two 
standards were reported The most frequently reported standards for advanced reading 
performance were also gains-based Forty-one percent of the secondary standards and 27% of 
the postsecondary standards were gains-based Thirty-two percent of the states did not indicate 
or had not determined standards for advanced reading at the postsecondary level, while only 1 7% 
had not developed or did not indicate a standard at the secondary level. 

Table 62 

Standards for Secondary and Postsecondary Reading Performance 



Post- 

Secondarv secondary 
Reading Performance f % f % 



0 



Basic 

Criterion-based standard 14 26 7 19 

Norm-based standard 12 23 8 22 

Gains-based standard 17 32 10 28 

Not determined/not indicated 10 19 11 31 

Advanced 

Criterion-based standard 10 22 5 23 

Norm-based standard 9 20 4 18 

Gains-based standard 19 41 6 27 

Not determined/not indicated 8 17 7 32 



Language 

Forty states at the secondary level and twenty-nine states at the postsecondary level 
assessed secondary basic language skill (see Table 1 and Table 32). Table 63 portrays infor- 
mation concerning the kinds of standards used by the states for basic and advanced language 
skill. Forty-eight standards were developed for secondary basic language performance. At the 
postsecondary level, thirty-six standards were reported. For basic language performance, the 
most frequently reported standards at the secondary level were gains-based (35%). Thirty-one 
percent of postsecondary standards were criterion-based, the most frequently reported kind of 
standard. 
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Thirty-five states at the secondary level and twenty-four states at the postsecondary level 
assessed advanced language skill (see Table 1 and Table 32). Forty-two standards were 
developed for secondary advanced language performance. At the postsecondary level, thirty-one 
standards were reported The most frequently reported standards for advanced language 
performance at the secondary level were gains-based (40%). The most frequently reported 
standards for advanced reading performance at the postsecondary level were criterion-based 
(39%). 



Table 63 



Standards for Secondary and Postsecondary Language Performance 



Language Performance 



Secondary 
f % 



Post- 
secondary 
f % 



Basic 



Criterion-based standard 
Norm-based standard 
Gains-based standard 
Not determined/not indicated 



13 
10 
17 
8 



27 
21 
35 
17 



11 

9 
9 
7 



31 
25 
25 
19 



Advanced 



Criterion-based standard 
Norm-based standard 
Gains-based standard 
Not determined/not indicated 



10 
9 
17 

6 



24 
21 
40 
14 



12 

6 
7 
6 



39 
19 
23 
19 



Mathematics 



Forty-six states at the secondary level and thirty states at the postsecondary level assessed 
basic mathematics skill (see Table 1 and Table 32). Table 64 portrays information concerning 
the kind of standard used by the states for basic and advanced mathematics skill. Fifty-six 
standards were developed for secondary basic mathematics performance. At the postsecondary 
level, forty standards were reported For basic mathematics performance, the most frequently 
reported standards at the secondary level were gains-based (38%). Thirty percent of 
postsecondary standards were criterion-based, the most frequently reported kind of standard. 
However, for postsecondary basic mathematics performance, 30% of the standards were not 
reported or not indicated. 
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Forty-one states at the secondary level and twenty-six states at the postsecondary level 
assessed advanced mathematics skill (see Table 1 and Table 32). Fifty-two standards were 
developed for secondary advanced mathematics performance. At the postsecondary level, twenty- 
nine standards were reported The most frequently reported standards for advanced language 
performance at the secondary level were gains-based (40%). The most frequently reported 
standards for advanced reading performance at the postsecondary level were criterion-based 
(31%). 



Table 64 

Standards for Secondary and Postsecondary Mathematics Performance 



Mathematics Performance 



Secondary 
f % 



Post- 

" xondary 
f % 



Basic 



Criterion-based standard 
Norm-based standard 
Gains-based standard 
Not determined/not indicated 



14 

13 
21 
8 



25 
23 
38 
14 



12 
7 
9 
12 



30 
17 
23 
30 



Advanced 



Criterion-based standard 
Norm-based standard 
Gains-based standard 
Not determined/not indicated 



12 
11 

21 
8 



23 
21 
40 
15 



9 
5 
7 
8 



31 
17 
24 
28 



Science 

Nineteen states at the secondary level and eight states at the postsecondary level assessed 
basic science skill (see Table 1 and Table 32). Table 65 portrays information concerning the 
kinds of standards used by the states for basic and advanced science skill. Twenty-six standards 
were developed for secondary basic science performance. At the postsecondary level, eight 
standards were reported For basic science performance, the most frequently reported standards 
at the secondary level were gains-based (37%) and criterion-based (37%). Fifty percent of 
postsecondary standards were gains-based, the most frequently reported kind of standard 
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Twenty-two states at the secondary level and eight states at the postsecondaiy level 
assessed advanced science skill (see Table 1 and Table 32). Twenty-eight standards were 
developed for secondary advanced science performance. At the postsecondaiy level, eight 
standards were reported The most frequently reported standards for advanced science perfor- 
mance at the secondary level were gains-based (40%). The most frequently reported standards 
for advanced science performance at the postsecondaiy level were gains-based (38%), Fifty 
percent of the standards at the postsecondaiy level were not determined or not indicated 

Table 65 

Standards for Secondary and Postsecondaiy Science Performance 



Post- 
Science Performance Secondary secondary 

f % f % 



0 



Basic 

Criterion-based standard 10 37 0 0 

Norm-based standard 4 15 1 12 

Gains-based standard 10 37 4 50 

Not determined/not indicated 3 11 3 38 

Advanced 

Criterion-based standard 10 36 0 0 

Norm-based standard 2 7 1 12 

Gains-based standard 11 40 3 38 

Not determined/not indicated 5 17 4 50 



Other Academic Performance 

Twenty-four states at the secondary level and twenty-seven states at the postsecondaiy 
level assessed other basic academic performance (see Table 1 and Table 32). Table 66 displays 
information concerning the land of standard used by the states for other basic and advanced 
academic performance. Thirty-two standards were developed for secondary other basic academic 
performance. At the postsecondaiy level, twenty-seven standards were reported. For other basic 
academic performance, the most frequently reported standards at the secondary level were 
criterion-based (41%). Thirty-three percent of postsecondaiy standards were criterion-based the 
most frequently reported kind of standard 
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Twenty-three states at the secondary level and twenty-four states at the postsecondary 
level assessed other advanced academic performance. Twenty-nine standards were developed for 
secondary other advanced academic performance. At the postsecondary level, twenty-four 
standards were reported. The most frequently reported standards for other advanced academic 
performance at the secondary level were gains-based (34%). The most frequently reported 
standards for other advanced academic performance at the postsecondary level were criterion- 
based (™- 



0. 



Table 66 

Standards for Secondary and Postsecondary Other Academic Performance 



Other Academic Performance 



Secondary 
f % 



Post- 
secondary 
f % 



Basic 



Criterion-based standard 
Norm-based standard 
Gains-based standard 
Not determined/not indicated 



13 
5 
8 
6 



41 
16 
25 
18 



9 
3 
5 
10 



33 
11 
19 

37 



Advanced 

Criterion-based standard 
Norm-based standard 
Gains-based standard 
Not determined/not indicated 



8 
4 
10 

7 



27 
14 
34 
24 



9 
2 
5 
8 



37 
8 
21 

33 



Other Performance 

In addition to developing standards for academic skills, states reported standards that were 
developed for competency attainment and work skill attainment. Information regarding the kinds 
of standards developed for these measures of other performance is contained in this section. 

Competency Attainme nt 

Twenty-four states at the secondary level and eighteen states at the postsecondary level 
assessed competency attainment (see Table 22 and Table 53). Table 67 contains information 
concerning the kinds of standards used by the states for assessing competency attainment. Thirty- 
seven standards were developed for secondary competency attainment. At the postsecondary 
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level, twenty-six standards were developed The most frequently reported standards at the 
secondary level were criterion-based (62%). Fifty-eight percent of postsecondary standards were 
criterion-based, the most frequently reported kind of standard 

Table 67 



Standards for Secondary and Postsecondary Competency Attainment 



Competency Attainment 


Secondary 
f 


% 


Post- 
secondarv 
f 


% 


Criterion-based standard 


23 


62 


15 


58 


Norm-based standard 


0 


0 


0 


0 


Gains-based standard 


1 


3 


1 


4 


Not determined not/indicated 


13 


35 


10 


38 



Work Skill Attainment 



Thirty-nine states at the secondary level and thirty-four states at the postsecondary level 
assessed work skill attainment (see Table 22 and Table 53). Table 68 displays information 
concerning the kinds of standards used by the states for assessing work skill attainment. Fifty- 
two standards were developed for secondary work skill attainment. At the postsecondary level, 
thirty-nine standards were developed The most frequently reported standards at the secondary 
level were also criterion-based (65%). Fifty-nine percent of postsecondary standards were 
criterion-based, the most frequently reported kind of standard 

Table 68 



Standards for Secondary and Postsecondary Work Skill Attainment 









Post- 




Work Skill Attainment 


Secondary 




secondarv 




f 


% 


f 


% 


Criterion based standard 


34 


65 


23 


59 


Norm-based standard 


1 


2 


1 


3 


Gain-based standard 


6 


12 


2 


5 


Not determined/not indicated 


11 


21 


13 


33 
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CONCLUSIONS 



Although some states chose to assess both basic and advanced academics with the same 
system oi performance measures and standards, sixteen states at the secondary level and nineteen 
states at the postsecondary level did develop separate systems for assessing basic and advanced 
academic skills. The type of assessment techniques used for assessing basic skills differed from 
those used in assessing advanced skills in those states that approved separate systems of 
performance measures and standards for basic and advanced academic skills. For example, in 
the assessment of secondary academic skills, high school proficiency exams were a commonly 
used technique for assessing basic academics, but less frequently used for advanced academics. 
Other techniques, such as portfolios and course/program completion were used more frequently 
to assess advanced academics than for basic academics. 

A number of nationally recognized instruments, such as the Iowa Test of Basic Skills were 
used to assess secondary basic academic skill in about one-third of the states: however, there was 
no consensus among the states on the type of nationally recognized instrument reported, eleven 
different instruments were approved for use in assessing basic academic skills by the states. 
Further, nationally recognized instruments were not the most popular method for assessing 
academic skill. About two-thirds of the states at the secondary level were using something other 
than nationally recognized techniques for assessing academic skills. 

If a gains-based standard was selected by a state as a standard for academic performance, 
local selected or nationally recognized techniques were favored over the use of state-developed 
assessment techniques. Using reading as an example, gains-based standards were the most popu- 
lar type of standard to be used with basic reading performance, followed by percent of students 
passing. For advanced academics, gains-based standards were the most used, with course/program 
completion being the next most frequently used type of standard 

Almost one-half of the states chose to develop a state wide instrument for assessing aca- 
demic skills or use an existing high school proficiency exam However, for assessing advanced 
academic skills, a wide variety of assessment techniques were reported 

Course/program completion was selected in a number of states as a standard of academic 
achievement. No information is currently available on a national basis concerning the local re- 
quirements for course/program completion or on the makeup of the specific courses and pro- 
grams. 

Of those measures and standards specifically listed in Perkins 1990, competency attain- 
ment was selected by the fewest number of states. The achievement of competency was the stan- 
dard most reported for competency attainment and, for the most part, states left it up to the local 
agencies to devise an assessment technique. This was probably die to the lack of existing instru- 
ments for assessing competency attainment. It appeared that states had difficulty in defining 
competency attainment. There was very little use of nationally recognized techniques for compe- 
tency attainment and competency attainment was assessed by course/program completion in a 
large number of states. 



Work skill attainment was predominately assessed by local selected or developed tech- 
niques at both the secondary and postsecondary level. The standard of choice for work skill at- 
tainment was a passing score on some type of an instrument designed to assess work skills. At 
both the secondary and postsecondary level, there was little use of nationally recognized tech- 
niques or licensure for work skill attainment. 

The most frequently reported standard for secondary program completion was 80-90% 
The average graduation rate of vocational students compared to the average for the total school 
population was the most frequently reported standard for the measure of high school graduation, 
using a standard of a 90-95% graduation rate. These findings tended to support the notion that 
states were requiring vocational students to perform at levels similar to the general high school 
population, in terms of school completion. 

For placement at the secondary level, the standards reported were higher for any place- 
ment than for related placement. This finding may be an indicator of the growing recognition 
among many that job-specific training is not as an important component of high school vocational 
education as a more general approach to employment training. 

The most frequently reported standards concerning service to special populations for both 
secondary and postsecondary programs were based on comparisons between numbers of special 
populations students enrolled and completing vocational programs and the numbers in the general 
school population. There were a variety of standards for assessing gender equity at both the sec- 
ondary or postsecondary level. 

For the most part, state developed techniques were not used for assessing postsecondary 
academic performance. Local selected and developed techniques and course/program completion 
were the top choices for assessing academic skills. There was no consensus among the states as 
to the types of standards to be used for assessing academic skills. The setting of standards was 
left to local agencies in a substantial number of states. 

Local selected and course/program completion were the most popular techniques for as- 
sessing competency attainment at the postsecondary level. Percent of students attaining compe- 
tency was the most popular standard 

Although licensure was reported in use more often in assessing work skill attainment in 
postsecondary programs then with secondary programs, local selected techniques were selected 
more often for postsecondary work skill attainment. In addition, the percent of students attaining 
passing scores on local developed or selected instruments was the most frequently reported stan- 
dard for postsecondary work skill attainment. There appeared to be wide variability across the 
states as to what was an acceptable standard for work skill attainment. 

There was also wide variability in the level of standards set for post secondary 
course/program completion. Course/program completion was the most frequently reported mea- 
sure of other performance, but there appeared to be little consensus on an appropriate level of 
completion. 
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Related placement rates tended to be set at higher levels for postsecondary than for sec- 
ondary programs. This could possibly be an indication that states were accepting the idea that 
highly specialized training was more appropriate at the postsecondary level, while allowing for 
more generalized training at the secondary level. 

Gain scores were more prevalent as a standard for secondary advanced reading than for 
basic reading skill or postsecondary advanced reading. More standards were not determined or 
not indicated at the post secondary level than at the secondary level. Criterion and norm based 
standards were used in approximately the same number of cases for secondary, and postsecondary 
advanced and basic reading. 

RECOMMENDATIONS 

The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 required 
each state to establish a system of performance measures and standards to use in the evaluation 
of vocational education programs. This study was conducted since little information was avail- 
able on the techniques and standards used by states in implementing these systems. The recom- 
mendations are based on the previous findings and conclusions and include the following: 

1. Information was not gathered concerning the rationale for adoption of standards. 
Research should be conducted to determine the rationale for selection of standards 
and the compatibility of state approved standards with business and industry stan- 
dards. 

2. Research should be conducted concerning the validity and reliability of state and lo- 
cal developed assessment techniques/instruments. 

3. Future legislation should clearly define competency attainment. 

4. Further development of instruments designed to assess work skill attainment and 
competency attainment should take place. 

5. Information is needed on the relationship between student outcomes on academic 
instruments developed specifically for vocational students and student outcomes on 
high school proficiency exams. 

6. A compendium of state and local developed assessment techniques should be devel- 
oped for vocational educators to use in selecting instruments and procedures. 

7. Further research should be conducted on the effectiveness of systems of standards 
and measures in achieving desired outcomes before any additional attempt is made 
in standardizing the measures and standards across the nation or in establishing levels 
of performance standards. 

The requirement for states to establish systems of performance measures and standards 
represented a continuation of the emphasis on evaluation that has been present in vocational 
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education legislation for many years. This study reports on the state systems that were in place 
as of March, 1993. This information is provided in an effort to assist those who are responsible 
for the continued development and implementation of these measures and standards. These state 
systems should be monitored for changes as they are developed 
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November 25, 1992 



I* 

Dear 2 ~, 

The implementation of the Carl D. Perkins Act of 1990 has required states to change how their students are 
being assessed and programs are being evaluated The statewide system of performance measures and 
standards was to be established by September, 1992. Many states also have revised their procedures for 
assessing program quality. Information on these recently approved standards and measures and the criteria 
used to assess program quality have not been summarized on a national basis. 

We are gathering this information from each state. We need your help in providing the following: 

* A list of the performance standards and measures for secondary and pos t seco ndat y vocational 
education that have been approved by your state board (Section 115). 

* The procedures you will use to make local modifications based on economic, geographic, or 
demographic factors, or the characteristics of the population to be served (Section 115). 

* Manuals, procedures, and criteria used to assess local program quality Section (1 16). 

* The uame of the individual(s) responsible for secondary and/or postsecondaty vocational 
education evaluation in each state. A one-page form has been included to provide us with 
this information. 

This information would be especially valuable in developing high quality vocational education programs and 
contributing to the study of vocational education's effectiveness. It would also be important in preparing new 
policies and legislation for vocational education. 

We need to receive this information by December 1, 1992. Thank you for your assistance in providing us with 
this information. We will provide you with a summary of the information we collect. 

Sincerely, 



N.L McCaslin 
Associate Professor 



William S. Headley 
Graduate Research Associate 
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March 11, 1993 
1~ 

Dear 2~ 

Thank you for sending us the information on the statewide system of measures and 
standards required by The Carl D. Perkins Act of 1990. In order to ensure the accuracy of 
the information we will be reporting, we are requesting that you review our findings for your 
state. 

Please check the summary sheet for measures adopted for use in secondary and post 
secondary programs* Feel free to make any necessary additions, deletions, or corrections. 
If the summary is correct, simply write "OK" on the summary sheet and return it to us. The 
enclosed instruction sheet will provide details to assist in the review of these materials. We 
would appreciate any comments concerning this summary. A stamped, addressed envelope 
is provided for your use in returning the summary forms. 

The final summary of the findings for all the states will be published and made available to 
you as soon as we have received the returned and corrected summary sheets. Therefore, 
we need to have this information by March 29, 1993. Don't hesitate to call if you have any 
questions. Thank you for your assistance in reviewing these materials. 

Sincerely, 

« 



N. L. McCaslin 
Associate Professor 



William S. Headley 
Graduate Research Associate 
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Instruction sheet 

1. Read over the attached summary sheet. An "x" has baan placad 
undar tha headings indicating your stata has a measure in this 
area. - . Blank spaces indicate no measure in this area. 

2. Please cross off any incorrect "x" and add any that should be 
present. 

3. Some states are phasing in measures, the measura is listed on 
the summary sheet even if it is to be phased in at a later 
date. 

4. The notes below will help in tha interpretation of the 
headings . 

a. If your stata shows an * in the academic skill area, we 
interpreted your system as using the same set of measures 

for both basic and advanced academic skill. 

b. Determined bv program means that each vocational program 
area (e.g. agriculture, marketing, business) in the state 
will determine the specific academic measures to be used. 

c. social studies includes citizenship, American history, 
etc. 

d. Under Placement , both Related and Ajjy include military 
service and further training or education. 

e. Under Special Populations and Other . Enrollment refers to 
measures of numbers of students in programs or ratios of 
students compared to other groups of students, etc. 

f* Program Features refers to measures of curriculum, 
teacher, or other programmatic measures. 

5. If the summary is correct, simply write "OK" on the summary 
sheet and return it to us. 

6. Please return the summary forms in the enclosed stamped, 
addressed envelope. 

7. Your comments are appreciated. If you need additional 
clarification or desire additional information, call Scot 
Headley at 614-292-6321. 
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